Unmanned aerial vehicle ISCC joint resource scheduling method and system based on lightweight DRL

Abstract

The application discloses a kind of unmanned aerial vehicle ISCC joint resource scheduling method and system based on light weight DRL, it is related to resource scheduling technical field.The application realizes the collaborative scheduling of communication, sensing and computing resources by unifying ISCC dynamic environment model and light weight D3PG-light network, overcomes the resource conflict caused by traditional module independence;Introducing LSTM and IFF module enhances the time sequence sensing and feature fusion capability, combined with AGS and segmented reward mechanism, high robustness can be maintained under channel mutation and task burst, the network parameter amount is compressed, the real-time deployment of DRL joint scheduling strategy is realized on light weight unmanned aerial vehicle platform, and the resource utilization, sensing accuracy and system stability are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource scheduling, in particular to a UAV ISCC joint resource scheduling method and system based on lightweight DRL. BACKGROUND

[0002] With the rapid development of wireless communication technology, intelligent sensing technology and edge computing technology, unmanned aerial vehicles (UAVs) have gradually become an important platform in low-altitude communication networks and intelligent service systems. Existing UAV communication systems usually rely on air-to-ground links (A2G) to provide data transmission services to ground users. A2G communication has characteristics such as complex path loss model, unstable shadow fading, and fast channel time variation, so it is necessary to improve the link reliability through bandwidth allocation, transmit power control, etc. At the same time, with the popularization of sensor technology, unmanned aerial vehicles can simultaneously perform image acquisition, target detection, environmental perception and other tasks, thereby realizing real-time monitoring of the ground scene. However, in the traditional architecture, the communication module and the perception module often run independently, resulting in resource isolation and insufficient scheduling efficiency, which cannot meet the real-time requirements of complex tasks.

[0003] To improve the utilization efficiency of spectrum and power resources, the academic community has proposed an integrated sensing and communication (ISAC) architecture, which uses unified hardware and spectrum resources to simultaneously realize wireless communication and environmental perception. However, current ISAC technology focuses more on waveform design and signal processing to improve the synergy gain, and less on mobile carrier scenarios with task dynamics and resource constraints, especially the resource conflict problem when communication, perception and computing tasks are concurrent on the unmanned aerial vehicle platform.

[0004] With the growing demand for edge computing and intelligent services, the concept of ISAC has been further expanded to integrated sensing, communication and computing (ISCC). An ISCC system not only needs to realize the synergy of communication and perception, but also needs to perform task processing, data offloading and edge computing under limited computing power, thereby forming a resource coupling relationship between communication links, perception tasks and computing tasks.

[0005] In existing research, unmanned aerial vehicles are often used as mobile edge nodes to provide computing offloading services for ground users. However, traditional ISCC research is usually based on the idealized assumption of independent modules, ignoring the mutual influence of communication, perception and computing tasks in terms of time scale and resource occupation. For example, communication bandwidth occupation may lead to a decrease in the performance of perception beam or computing task scheduling; too high perception resources may reduce data transmission capacity; and user task queues may accumulate, thereby affecting the overall latency of the system. Therefore, how to model and cooperatively schedule communication, perception and computing tasks in a shared resource environment has become a key technical challenge in ISCC systems.

[0006] Multi-access edge computing (MEC) technology can migrate part of the computing task from the ground user to the edge node closer to the application scene to execute, thereby reducing the transmission delay and improving the service quality. In the unmanned aerial vehicle platform, the MEC technology is often used to process image recognition, data filtering, lightweight inference and other computing tasks.

[0007] However, the computing capability, energy reserve and load capacity of the unmanned aerial vehicle itself are very limited, and cannot support a too large neural network model or a high-load operation task. Therefore, the research focus of the UAV-MEC system is concentrated on: task offloading strategy; unmanned aerial vehicle CPU frequency control; task processing order; service queue stability. However, these researches generally ignore the influence of perception tasks and communication load, and lack a complete system modeling for the ISCC scene.

[0008] Deep reinforcement learning (DRL) has been widely used in unmanned aerial vehicle trajectory planning, power control, task offloading and channel adaptation. DRL can learn the optimal strategy in a complex dynamic scene through interaction with the environment, and is suitable for high-dimensional decision problems that cannot be solved by analytical methods. The common DRL algorithms at present include: deep deterministic policy gradient (DDPG), double-delay deep deterministic policy gradient (TD3), advantage actor-critic (A2C / A3C) and proximal policy optimization (PPO). These algorithms perform well in static or weakly time-varying communication scenarios, but there are two outstanding problems in the unmanned aerial vehicle ISCC scene: training instability: the strong randomness of the A2G channel can cause serious deviation of the Q value estimation; the model size is too large: the parameter quantity of the mainstream DRL network is dozens of ten thousand, which is difficult to deploy on the lightweight hardware of the unmanned aerial vehicle; lack of time series modeling: there is obvious time dependence between the link change and the queue dynamics, and it is difficult to capture the time sequence features only by relying on the MLP (multi-layer perception); the coupling between tasks is complex: there is a nonlinear relationship between the communication rate, the perception accuracy and the computing delay, and the traditional DRL is difficult to learn directly. SUMMARY

[0009] The purpose of the present application is to provide a lightweight DRL-based unmanned aerial vehicle ISCC joint resource scheduling method and system to improve the above technical problems.

[0010] In order to achieve the above invention purpose, the embodiment of the present application provides the following technical scheme:

[0011] A lightweight DRL-based unmanned aerial vehicle ISCC joint resource scheduling method, comprising:

[0012] collecting UAV multi-dimensional scheduling data, tasks, task queue states of a current time slot and perception-computation feedback of a previous time slot to generate a state vector of the current time slot; the UAV multi-dimensional scheduling data includes communication data and real A2G link measurement data of the UAV; the tasks include perception tasks, computation tasks and communication tasks;

[0013] constructing a D3PG-Light model based on an IFF feature fusion mechanism, an LSTM network and an MLP;

[0014] processing the state vector through the D3PG-Light model to generate an initial resource allocation scheme and a state value of the current time slot;

[0015] calculating task delays and multi-target rewards by simulating the initial resource allocation scheme by the UAV; updating the D3PG-Light model based on the multi-target rewards and performing iteration to generate a final resource allocation scheme; the task delays include communication delays, computation delays and perception delays.

[0016] In the above scheme, the D3PG-Light model based on IFF feature fusion, LSTM time series modeling and light-weight MLP is constructed to realize unified state representation and joint resource scheduling of communication, perception and computation tasks, and solve the key problems of resource fragmentation, scheduling conflict and difficulty in lightweight deployment of the model in the traditional ISCC system; the light-weight network structure and multi-target segmented reward mechanism support real-time operation on an embedded platform, significantly reduce the end-side inference delay and energy consumption, and improve the time series adaptation capability and multi-target collaborative optimization performance of the system under dynamic channels and burst tasks.

[0017] Further, the state vector of the current time slot is generated, including:

[0018] collecting UAV multi-dimensional scheduling data of a current time slot, setting a time slot length and ISCC parameters, and constructing an ISCC environment;

[0019] calculating channel calculation data of a channel gain sequence in the real A2G link measurement data by using a sliding window length;

[0020] calculating a signal-to-noise ratio and a communication rate by using a clip function and a Shannon formula based on the real A2G link measurement data;

[0021] determining a UAV communication distance and a flight height based on position information of the UAV, and generating A2G channel statistical features based on the channel calculation data, the signal-to-noise ratio and the communication rate;

[0022] updating task queue states corresponding to communication tasks and computation tasks based on a communication data amount and a computation task amount of the UAV in the current time slot;

[0023] obtain the perception-computation feedback of the previous time slot and the task of the UAV in the current time slot; and construct a state vector of the current time slot based on the updated task queue state and the A2G altitude statistical characteristics.

[0024] Further, the D3PG-Light model comprises an IFF feature fusion module, an LSTM module, an Actor network and a Critic network connected in series; the Actor network and the Critic network each comprise a first MLP layer and a second MLP layer.

[0025] The IFF feature fusion module comprises a channel feature branch, a queue feature branch, a feature concatenation layer, a third lightweight linear layer and a third activation function layer; the channel feature branch comprises a first lightweight linear layer and a first activation function layer; the queue feature branch comprises a second lightweight linear layer and a second activation function layer.

[0026] Further, the generating of the initial resource allocation scheme and the state value of the current time slot comprises:

[0027] inputting the state vector into the IFF feature fusion module to generate communication-queue fusion features of the current time slot;

[0028] inputting the communication-queue fusion features into the LSTM module for time evolution to generate channel-queue time sequence features of the current time slot;

[0029] inputting the channel-queue time sequence features of the current time slot into the Actor network to generate a resource allocation action of the current time slot, i.e. to obtain the initial resource allocation scheme;

[0030] merging the initial resource allocation scheme and the channel-queue time sequence features and inputting them into the Critic network to calculate the state value of the current time slot.

[0031] Further, the generating of the final resource allocation scheme comprises:

[0032] simulating the execution of the initial resource allocation scheme by the UAV, calculating the communication delay, the computation delay, the perception delay and the perception accuracy index based on the communication rate, the computation task and the task queue state corresponding to the communication task in the current time slot;

[0033] calculating a multi-objective reward based on the communication delay, the computation delay, the perception delay and the perception accuracy index; the multi-objective reward comprises a total delay cost, a perception accuracy reward, a throughput efficiency reward, a resource occupation penalty, an energy consumption penalty, a cascading extreme state penalty and an action smoothing penalty;

[0034] updating the D3PG-Light model based on the multi-objective reward and the state value of the current time slot through an adaptive gradient stabilization mechanism and Q value clipping.

[0035] updating the task queue state corresponding to the communication task and the computing task; and iteratively using the updated D3PG-Light model until a stable convergence condition is met to generate a final resource allocation scheme.

[0036] In the above scheme, the present application overcomes the problem of unstable training and difficult convergence of traditional reinforcement learning in the ISCC scenario of the UAV; by simultaneously optimizing the communication delay, computing delay, perception delay and perception accuracy, and introducing cascade extreme state punishment and action smoothing punishment in the reward function, the problem of policy shock and resource allocation fluctuation under multi-objective conflict is effectively solved; combined with adaptive gradient stabilization and Q value clamping mechanism, the gradient explosion and value overestimation in the training process are further suppressed, and the training stability and convergence reliability of the algorithm on the lightweight embedded platform are improved, so that the scheduling strategy is efficiently, stably and adaptively optimized in the real dynamic environment while ensuring the lightweight of the model.

[0037] Further, the computing of the communication delay, the computing delay, the perception delay and the perception accuracy indicators comprises:

[0038] computing the communication delay based on the communication rate of the current time slot and the task queue state corresponding to the communication task;

[0039] computing the computing delay based on the task size and the task queue state of each computing task in the current time slot;

[0040] computing the perception delay based on the reference perception time constant and the signal-to-noise ratio of the current time slot through a function with marginal decreasing characteristics;

[0041] constructing a perception accuracy model through an S-shaped activation function, and computing the perception accuracy indicator based on the signal-to-noise ratio of the current time slot through the perception accuracy model.

[0042] In the above scheme, the present application solves the key problems such as delay estimation distortion, perception performance modeling simplification and multi-objective optimization coupling difficulty in the traditional UAV ISCC scheduling by introducing structured delay and accuracy calculation models: first, the communication and computing delays are split into inherent processing delay and queuing waiting delay, which corrects the unreasonable situation that the total delay tends to zero when the queue is empty, and provides accurate performance feedback for resource scheduling; at the same time, the perception delay is modeled by a marginal decreasing function (describing the delay law of the joint action of resources and channels), and the perception accuracy is described by an S-shaped function (reflecting the nonlinear characteristics of the rapid rise and then saturation of accuracy under resource constraints), which is consistent with the actual behavior of the sensing system to provide reliable perception performance characterization, and fundamentally improves the overall system performance and resource utilization efficiency.

[0043] Further, the computing of the multi-objective reward comprises:

[0044] based on the communication delay, the calculation delay and the perception delay, the total delay cost is calculated by a log function;

[0045] based on the perception accuracy index, the perception accuracy reward is calculated by a monotonically increasing function;

[0046] based on the communication data volume of each communication task, the average communication data volume is calculated, and the throughput efficiency reward is calculated in combination with the task service volume of the communication task;

[0047] based on the initial resource allocation scheme, the resource occupation penalty is calculated;

[0048] based on the transmission power, the calculation task volume completed by the UAV and the perception occupied resource, the energy consumption penalty is calculated;

[0049] based on the preset delay threshold, the resource occupation threshold and the energy consumption threshold, in combination with the initial total delay cost, the energy consumption penalty and the initial resource allocation scheme, the cascaded extreme state penalty is calculated;

[0050] based on the initial resource allocation scheme of the current time slot and the last time slot, the action smoothing penalty is calculated;

[0051] based on the total delay cost, the perception accuracy reward, the throughput efficiency reward, the resource occupation penalty, the energy consumption penalty, the cascaded extreme state penalty and the action smoothing penalty, the multi-objective reward is calculated by weighted fusion.

[0052] In the above scheme, the present application solves the problems of multi-objective difficult to balance, extreme state lacking of constraint and strategy easy to shock in the traditional reward design by constructing a multi-objective segmented reward function that integrates the total delay cost, the perception accuracy reward, the throughput efficiency reward, the resource occupation penalty, the energy consumption penalty, the cascaded extreme state penalty and the action smoothing penalty. The design enables the agent to simultaneously optimize the delay, the accuracy, the throughput, the energy consumption and the stability, and realizes adaptive optimization through a dynamic weighting mechanism, significantly improving the balance, robustness and overall system performance of the strategy.

[0053] A UAV ISCC joint resource scheduling system based on a light DRL includes:

[0054] A vector generation module is configured to collect UAV multi-dimensional scheduling data of a current time slot, tasks, task queue states and perception-computation feedback of a last time slot, and generate a state vector of the current time slot; the UAV multi-dimensional scheduling data includes communication data and real A2G link measurement data of the UAV; the tasks include perception tasks, calculation tasks and communication tasks;

[0055] A construction module is configured to construct a D3PG-Light model based on an IFF feature fusion mechanism, an LSTM network and an MLP;

[0056] The scheme initial generation module is configured to process the state vector by the D3PG-Light model to generate an initial resource allocation scheme and a state value of a current time slot.

[0057] The scheme final generation module is configured to execute the initial resource allocation scheme by the UAV, calculate a task delay and a multi-target reward, update the D3PG-Light model and perform iteration based on the multi-target reward, and generate a final resource allocation scheme. The task delay includes a communication delay, a calculation delay and a perception delay.

[0058] In the above scheme, the system realizes unified scheduling and dynamic optimization of communication, perception and calculation resources, solves the problems of resource fragmentation, serious conflict and difficulty in model deployment in the traditional architecture. Through the lightweight network design, the parameter quantity is reduced by about 97%, supporting ≤10ms real-time inference on an embedded platform, meeting the high-frequency decision-making requirements of the UAV. The system has strong timing adaptability, maintains high robustness in a dynamic environment, and optimizes the delay, accuracy, energy consumption and throughput through a multi-target segmented reward mechanism, significantly improving the overall scheduling performance and resource utilization. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0060] Figure 1 The method flowchart in the embodiment 1 of the present application;

[0061] Figure 2 The D3PG-Light model structure diagram in the embodiment 1 of the present application;

[0062] Figure 3 The IFF feature fusion module structure diagram in the embodiment 1 of the present application;

[0063] Figure 4 The system structure diagram in the embodiment 1 of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0065] Embodiment 1

[0066] Please refer to Figure 1 The embodiment provides a UAV ISCC joint resource scheduling method and system based on lightweight DRL. Figure 1 The execution subject of the method can be a software and / or hardware device. The execution subject of the present application can include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, the user equipment can include but is not limited to computers, smart phones, personal digital assistants (Personal Digital Assistant, PDA) and the above-mentioned electronic devices, etc. The network equipment can include but is not limited to a single network server, a server group composed of multiple network servers or a cloud composed of a large number of computers or network servers based on cloud computing, wherein cloud computing is a kind of distributed computing, which is a super virtual computer composed of a group of loosely coupled computers. The present embodiment does not make any limitation.

[0067] S1, collect the UAV multi-dimensional scheduling data, tasks, task queue state of the current time slot and the perception-computation feedback of the last time slot, and generate the state vector of the current time slot; the UAV multi-dimensional scheduling data includes the communication data of the UAV and the real A2G link measurement data; the tasks include perception tasks, computation tasks and communication tasks;

[0068] The S1 includes:

[0069] S1-1, collect the UAV multi-dimensional scheduling data of the current time slot, set the time slot length and the ISCC parameter, and construct the ISCC environment;

[0070] In the embodiment, the time slot length is 0.01s. The ISCC parameter includes the total bandwidth, the thermal noise power spectral density, the maximum transmission power of the UAV, the total amount of MEC computing resources and the lower limit of MEC computing resources. The total bandwidth is 5MHz, and the thermal noise power spectral density The calculation formula is: The maximum transmit power of the UAV is 2.0W, and the total amount of MEC computing resources is 5x10 9 FLOPs / s, and the lower limit of MEC computing resources is 1x10 6 FLOPs / s. represents the Boltzmann constant, represents the thermodynamic temperature,

[0071] According to the ISCC parameters, the corresponding ISCC environment is constructed, so that the D3PG-Light model can be trained and iterated under clear and controllable physical boundaries and resource constraints.

[0072] S1-2, calculate channel calculation data of channel gain sequence in real A2G link measurement data by using sliding window length; the channel calculation data includes sliding mean, channel sliding standard deviation, channel sliding maximum value and channel sliding minimum value.

[0073] Preferably, the formula corresponding to the channel sliding mean is:

[0074] ;

[0075] wherein, represents the absolute value, represents the channel gain sequence of the i-th time slot, represents the sliding window length, represents the channel sliding mean, represents the time slot index value, represents the summation function.

[0076] Calculate the channel sliding standard deviation based on the channel sliding mean, and select the corresponding maximum value and minimum value in the channel gain sequence.

[0077] S1-3, based on real A2G link measurement data, calculate signal-to-noise ratio and communication rate by clip function and Shannon formula;

[0078] Specifically, when the resource actually takes effect, the coupling of light communication and perception is introduced, that is, the higher the perception resource, the slightly reduced available bandwidth, which embodies the physical constraints of co-platform and co-power in the ISCC scenario. Therefore, the effective bandwidth ratio is:

[0079] ;

[0080] wherein, represents the upper and lower bound truncation, ​Allocate the proportion of sensing resources required for the sensing task in the current time slot (sensing resource proportion). This indicates the proportion of communication bandwidth required by the communication task in the current time slot (communication bandwidth percentage).

[0081] Therefore, the effective communication bandwidth for:

[0082] ;

[0083] in, To ensure bandwidth limits are met and to prevent excessively low noise power; It is a function for maximizing the value;

[0084] Signal-to-noise ratio (SNR) for:

[0085] ;

[0086] According to Shannon's formula, communication rate for:

[0087] ;

[0088] in, For transmission power, This is a neglect of the constant base 2.

[0089] S1-4. Determine the UAV communication distance and flight altitude based on the UAV's location information, and generate A2G channel statistical characteristics based on channel calculation data, signal-to-noise ratio, and communication rate;

[0090] S1-5. Based on the amount of communication data and computation tasks of the UAV in the current time slot, update the task queue status corresponding to the communication tasks and computation tasks.

[0091] Specifically, since discrete backlog queues are established only for communication and computation tasks, and perception tasks do not form discrete queuing structures and do not participate in queue evolution, only the task queue states of communication and computation tasks need to be updated. The corresponding formula is:

[0092] ;

[0093] ;

[0094] in, , These represent the current time slot and the previous time slot, respectively. The task queue status corresponding to the communication task. , These represent the previous time slot. a task service amount of a communication task in the last time slot, , respectively represent the task queue state corresponding to the computing task in the current time slot and the last time slot, , , respectively represent the task service amount and the task amount of the computing task in the last time slot.

[0095] The task service amount is determined by the allocated bandwidth, transmission power and computing resources, for example, the formula corresponding to the task service amount of the communication task is:

[0096] ;

[0097] The formula corresponding to the task service amount of the computing task is:

[0098] ;

[0099] wherein, represents the proportion of resources required for the computing task in the last time slot, represents the communication rate in the last time slot.

[0100] S1-6, obtain the perception-computing feedback in the last time slot and the task of the UAV in the current time slot; based on the updated task queue state and the A2G height statistical characteristics, construct a state vector of the current time slot. The state vector includes 9 types of feature vectors, and the expression is:

[0101] ;

[0102] wherein, , , , respectively represent the channel sliding mean, channel sliding standard deviation, channel sliding maximum and channel sliding maximum in the current time slot , , respectively represent the normalized communication distance and flight height of the UAV in the current time slot , , respectively represent the normalized queue length of the communication task and the computing task in the current time slot .

[0103] Preferably, the computing task and the communication task are subject to Poisson distribution arrival, that is:

[0104] ;

[0105] ​ ;

[0106] wherein, , respectively represent the amount of communication data and the amount of computing tasks arrived in the first time slot, , , both represent the average arrival rate, represents a Poisson distribution.

[0107] The perception task adopts a continuous flow model rather than a discrete Poisson arrival, that is, the perception operation is performed in each time slot, and the perception workload is regarded as a fixed reference load per unit time The amount of arrival of the perception task does not form a discrete backlog queue, and the processing load thereof is:

[0108] ;

[0109] wherein, represents the processing load of the perception task.

[0110] S2, based on the IFF feature fusion mechanism, the LSTM network and the MLP, a D3PG-Light model is constructed;

[0111] Preferably, as shown in Figure 2 , the D3PG-Light model comprises an IFF feature fusion module, an LSTM module, an Actor network and a Critic network connected in series; the Actor network and the Critic network each comprise two layers of MLP (Multi-Layer Perceptron), namely a first MLP layer and a second MLP layer; as shown in Figure 3 , the IFF feature fusion module comprises a channel feature branch, a queue feature branch, a feature splicing layer, a third lightweight linear layer and a third activation function layer; the channel feature branch comprises a first lightweight linear layer and a first activation function layer; the queue feature branch comprises a second lightweight linear layer and a second activation function layer. The first activation function layer, the second activation function layer and the second activation function layer each adopt a ReLU function. The first lightweight linear layer to the third lightweight linear layer are each an FC layer (Full Connection layer).

[0112] The traditional multi-layer linear structure (such as a fully connected network) usually adopts a high-dimensional hidden layer design such as 128 or 256, which can effectively realize the nonlinear mapping of features and information fusion, but needs a large number of network parameters and computing overhead, is easy to cause the model to be bloated, the reasoning delay to be high, the energy consumption to be large, and is difficult to be deployed to a lightweight hardware platform such as a UAV. Therefore, the present application adopts a lightweight linear layer design to construct an IFF feature fusion module. The output dimensions of the first lightweight linear layer to the third lightweight linear layer are 64, 32 and 96 respectively, and are input to a subsequent lightweight LSTM for time series modeling, which significantly reduces the parameter quantity and computing complexity of the linear layer, and realizes lightweight heterogeneous feature fusion.

[0113] In the IFF feature fusion module, the input data (state vector) is classified and split into a channel-related sub-vector and a queue-related sub-vector, and is input to a channel feature branch and a queue feature branch respectively. The feature vectors output by the channel feature branch and the queue feature branch are spliced and linearized to generate output data (communication-queue fusion features) of the IFF feature fusion module, and are transmitted to an Actor network and a Critic network respectively. The expression of the channel-related sub-vector is as follows:

[0114] ;

[0115] The expression of the queue-related sub-vector is as follows:

[0116] .

[0117] In the channel feature branch, the channel-related sub-vector is input to the first lightweight linear layer to generate corresponding channel lightweight linear features, which are nonlinearly transformed by a ReLU function to generate 64-dimensional channel initial features , and the corresponding formula is as follows:

[0118] ;

[0119] wherein, , respectively represent the weight and bias of the first lightweight linear layer, represents the ReLU function. The channel feature branch and the queue feature branch are the same in other processing processes except that the dimensions of the input / output data are different.

[0120] S3, processing the state vector by the D3PG-Light model to generate an initial resource allocation scheme and a state value of the current time slot;

[0121] The S3 includes:

[0122] S3-1, input the state vector into the IFF feature fusion module to generate the communication-queue fusion feature of the current time slot;

[0123] S3-2, input the communication-queue fusion feature into the LSTM module for time sequence evolution to generate the channel-queue time sequence feature of the current time slot;

[0124] Specifically, the low-dimensional communication-queue fusion feature output by the IFF feature fusion module is input into the LSTM module to realize time sequence modeling of channel dynamics and queue evolution, and the corresponding formula is:

[0125] ;

[0126] wherein, denotes the communication-queue fusion feature, , denote the hidden state (channel-queue time sequence feature) of the previous time slot and the current time slot, respectively, , denote the LSTM memory state of the previous time slot and the current time slot, respectively, denotes the LSTM network.

[0127] In this embodiment, compared with the conventional time sequence modeling method using multi-layer LSTM, which has large parameter quantity and heavy calculation burden, the single-layer LSTM structure is adopted in this embodiment, which can significantly reduce the parameter quantity by about 70% while keeping the sensitivity to the time sequence features of the channel state and the task queue dynamics, thereby greatly reducing the calculation and storage overhead of the model. In addition, the single-layer LSTM structure provides a special time sequence embedding representation space for the fused heterogeneous features, which not only reduces the overall learning difficulty of the network, but also enhances the understanding ability of the intelligent agent to the internal correlation of the communication-sensing-computing joint features.

[0128] S3-3, input the channel-queue time sequence feature of the current time slot into the Actor network to generate the resource allocation action of the current time slot , that is, to obtain the initial resource allocation scheme; the expression corresponding to the initial resource allocation scheme is:

[0129] ;

[0130] wherein, denotes the proportion allocation of the communication bandwidth required by the communication task under the current time slot, denotes the proportion allocation of the sensing resource required by the sensing task under the current time slot, denotes the proportion allocation of the computing resource required by the computing task under the current time slot.

[0131] Specifically, the formula corresponding to the Actor network is:

[0132] ;

[0133] wherein, represents an activation function, , respectively represent the weight and bias term of the first MLP layer, , respectively represent the weight and bias term of the second MLP layer. The resource allocation action is linearly scaled to [0, 1] 3 .

[0134] In addition, in order to avoid the communication rate tending to zero caused by extremely low bandwidth, a constraint is imposed on the bandwidth allocation:

[0135] ;

[0136] wherein, represents the minimum value of the proportion of the allocated . In the embodiment, the value of is 0.1, which can significantly improve the stability of the system.

[0137] S3-4, merge the initial resource allocation scheme and the channel-queue timing characteristics and input into the Critic network to calculate the state value of the current time slot , and the corresponding formula is:

[0138] ;

[0139] wherein, , respectively represent the weight and bias term of the first MLP in the Critic network, , respectively represent the weight and bias term of the second MLP in the Critic network, represents the fusion data obtained by merging the initial resource allocation scheme and the channel-queue timing characteristics.

[0140] S4, execute the initial resource allocation scheme by simulating the UAV, calculate the task delay and multi-objective reward; based on the multi-objective reward, update the D3PG-Light model and iterate to generate a final resource allocation scheme; the task delay includes communication delay, calculation delay and perception delay.

[0141] The S4 includes:

[0142] S4-1, performing an initial resource allocation scheme by a UAV, calculating a communication delay, a computing delay, a perception delay and a perception accuracy index based on a communication rate of a current time slot, a computing task and a task queue state corresponding to a communication task;

[0143] The S4-1 comprises:

[0144] S4-1-1, calculating a communication delay based on a communication rate of a current time slot and a task queue state corresponding to a communication task;

[0145] Specifically, the communication delay comprises an inherent communication delay and a queuing waiting delay.

[0146] The inherent communication delay The corresponding formula is:

[0147] ;

[0148] The queuing waiting delay The corresponding formula is:

[0149] ;

[0150] The communication delay The corresponding formula is:

[0151] ;

[0152] wherein, denotes a data packet size of a communication task processing, denotes a small constant for preventing division by zero.

[0153] S4-1-2, calculating a computing delay based on a task size of each computing task of a current time slot and a task queue state;

[0154] Specifically, taking any one computing task as an example, the computing delay comprises an inherent computing delay and a queuing waiting delay.

[0155] The inherent computing delay The corresponding formula is:

[0156] ;

[0157] ;

[0158] The queuing waiting delay The corresponding formula is:

[0159] ;

[0160] Further, the computing delay The corresponding formula is:

[0161] ;

[0162] wherein, represents a small constant preventing division by zero, represents the MEC computing capability, represents the size of a single computing task, represents the proportion of the current time slot allocated to the computing task required resources.

[0163] S4-1-3, based on the benchmark perception time constant and the signal-to-noise ratio of the current time slot, the proportion of the perception task required perception resources is allocated, and the perception time delay is calculated through a function with marginal decreasing characteristics; the function with marginal decreasing characteristics corresponds to the formula:

[0164] ;

[0165] wherein, represents the SNR gain factor, represents the perception time delay, represents the benchmark perception time constant. The function reflects the dual effects of perception resource investment and link quality on time delay improvement, and naturally produces marginal reduction effect.

[0166] S4-1-4, the perception accuracy model is constructed through the S-shaped activation function, based on the signal-to-noise ratio of the current time slot and the proportion of the perception task required perception resources is allocated, the perception accuracy index is calculated through the perception accuracy model;

[0167] Specifically, the perception task of the unmanned aerial vehicle, such as target recognition and environment detection, is sensitive to resources and channel quality, and the embodiment adopts S-shaped activation function (Sigmoid) to construct the perception accuracy model, so that the accuracy is close to 0 when the resource is too low, the accuracy rises rapidly when the resource is moderate, and the accuracy gradually saturates when the resource is sufficient. Therefore, the perception accuracy model The corresponding formula is:

[0168] ;

[0169] wherein, represents the perception accuracy index, represents the exponential function with base e, , respectively represent the shape parameters of the control curve slope and the threshold value, both shape parameters are greater than 0.

[0170] In this embodiment, by explicitly splitting the inherent delay and queuing delay of the computing task and the communication task, and introducing a nonlinear perception latency and accuracy model, the unrealistic assumptions that the latency approaches 0 when the queue is empty and the perception task does not change with resources can be overcome, so that the scheduling strategy can be jointly optimized around the real system indicators, and the latency performance and perception quality of the UAV ISCC system are improved.

[0171] S4-2, based on the communication latency, computing latency, perception latency and perception accuracy indicators, calculate the multi-objective reward; the multi-objective reward includes total delay cost, perception accuracy reward, throughput efficiency reward, resource occupation penalty, energy consumption penalty, cascade extreme state penalty and action smoothing penalty;

[0172] The S4-2 includes:

[0173] S4-2-1, based on the communication latency, computing latency and perception latency, calculate the total delay cost by the log function;

[0174] The formula corresponding to the S4-2 is:

[0175] ;

[0176] ;

[0177] wherein, represents the initial total delay cost, represents the total delay cost. 100 is a normalized reference scale for keeping the time delays of different dimensions comparable at the same scale.

[0178] S4-2-2, based on the perception accuracy indicator, calculate the perception accuracy reward by a monotonically increasing function , the corresponding formula is:

[0179] ;

[0180] wherein, represents a monotonically increasing function for mapping the physical layer perception accuracy to a reinforcement learning reward signal. In this embodiment, the monotonically increasing function can adopt:

[0181] ;

[0182] In the perception task, the higher the perception accuracy, the better the performance of the system, so in the reward function, the perception accuracy and the reward should have a consistent positive relationship, that is, a monotonically increasing function is adopted.

[0183] S4-2-2, based on the communication data volume of each communication task, calculate the average communication data volume, and combine the task service volume of the communication task to calculate the throughput efficiency reward;

[0184] Specifically, the ratio of the average communication data volume and the task service volume of the communication task can reflect the queuing stability of the link, so that the ratio is selected as the throughput efficiency reward, which can be used to encourage the communication task to be digested as much as possible under the premise of ensuring controllable delay, and improve the queue stability and effective throughput. Further, the throughput efficiency reward The corresponding formula is:

[0185] ;

[0186] Among them, The average communication data volume of the communication task is represented.

[0187] S4-2-3, based on the initial resource allocation scheme, calculate the resource occupation penalty The corresponding formula is:

[0188] ;

[0189] Among them, The resource occupation penalty coefficient is represented, which is used to avoid unnecessary energy consumption and system load caused by excessive resource allocation.

[0190] S4-2-4, based on the transmission power, the amount of calculation task completed by the UAV, and the sensing occupied resource, calculate the energy consumption penalty.

[0191] Specifically, the communication energy consumption is calculated by the transmission power The calculation energy consumption is calculated by the amount of calculation task completed by the UAV The sensing energy consumption is calculated by the sensing occupied resource The sum of the three kinds of energy consumption is calculated to generate the energy consumption penalty The corresponding formula is:

[0192] ;

[0193] Among them, The energy consumption penalty coefficient is represented. In this embodiment, since the ways of calculating communication energy consumption, calculation energy consumption and sensing energy consumption are prior art, they are not described in detail.

[0194] S4-2-5, based on the preset delay threshold, resource occupation threshold and energy consumption threshold, combined with the initial total delay cost, energy consumption penalty and initial resource allocation scheme, calculate the cascade extreme state penalty;

[0195] Specifically, when the system simultaneously appears high delay high resource occupation or high delay high energy consumption and other extreme situations, an additional cascade penalty is applied, that is, a cascade extreme state penalty is set, which is used to prevent the strategy from entering a catastrophic state of slow and high consumption, and improve the system operation safety. The cascade extreme state penalty The corresponding formula is:

[0196] ;

[0197] wherein, , , respectively represent the punishment intensity in different extreme states, , , represent the delay threshold, , respectively represent the resource occupation threshold and the energy consumption threshold.

[0198] S4-2-6, based on the initial resource allocation scheme of the current time slot and the last time slot, calculate the action smoothness punishment;

[0199] Specifically, the difference degree of the two initial resource allocation schemes of adjacent time slots , is defined as:

[0200] ;

[0201] The action smoothness punishment is used to suppress the rapid oscillation of resource allocation and improve the execution stability in online deployment, and the corresponding formula is:

[0202] ;

[0203] wherein, represents the action smoothness punishment coefficient, represents the norm 2 (Euclidean distance).

[0204] S4-2-7, based on the total delay cost, the perception accuracy reward, the throughput efficiency reward, the resource occupation punishment, the energy consumption punishment, the cascade extreme state punishment and the action smoothness punishment, calculate the multi-objective reward by weighted fusion.

[0205] Specifically, in order to make the UAV obtain the overall optimal performance among the three types of sub-tasks of communication, calculation and perception, based on the multi-objective reward function of "delay-accuracy-throughput-energy consumption-stability", the communication delay, the calculation delay and the perception delay are calculated in each time slot, and additional indexes such as resource occupation, energy consumption and action smoothness are introduced, and the multi-objective reward is calculated by weighted method , and the corresponding formula is:

[0206] ;

[0207] wherein, represents the delay weight; represents the perception accuracy weight; ​Indicates the throughput efficiency weight. The three weights can be dynamically adjusted according to the queue pressure and channel conditions.

[0208] S4-3, based on the current time slot multi-objective reward and state value, through the adaptive gradient stabilization mechanism (AGS) and Q value clamp (Q-Clamp stabilization strategy), update the D3PG-Light model;

[0209] Specifically, in deep reinforcement learning, due to the problems of gradient explosion, training shock and the like caused by large environmental noise and dramatic changes in reward scale, an adaptive gradient stabilization (AGS) mechanism is added to the parameter update process of the Actor and Critic networks to dynamically constrain the gradient obtained by each backpropagation.

[0210] In order to avoid over-optimistic estimation or value divergence of the Critic in training, a Q value clamp (Q-Clamp) mechanism is introduced at the output end of the value network to limit all Q values within a reasonable numerical range, avoid gradient explosion, suppress overestimation of value, and ensure the stability and reliability of value update.

[0211] Therefore, the processing procedure of S4-3 is as follows:

[0212] First, a batch of historical interaction data is sampled from the experience pool, including the state vector , action , multi-objective reward , and the state vector of the next iteration. An actor network is used to generate a recommended action according to , and a critic network is used to evaluate the value of the state-action pair to obtain the original target Q value . To prevent value estimation divergence, the is limited within a preset interval by a Q-Clamp mechanism, that is, according to the formula:

[0213] ;

[0214] The clamped Q value is obtained.

[0215] Based on the clamped Q value, combined with the discount factor and the termination flag, according to the formula:

[0216] ;

[0217] The time difference target is constructed, which represents the value that the current decision should obtain in a long-term perspective and is the benchmark for the Critic network learning. Among them, , min and max represent the minimum and maximum value of Q respectively; gamma represents the discount factor, controlling the weight of future rewards in the current value; done represents the terminal flag, 1 represents the terminal state, 0 represents the non-terminal state.

[0218] The state vector of the current iteration , action is input into the Critic network to generate the value prediction of the current iteration , which is clamped in the same way as the raw target Q value to obtain the value prediction clamp value . Then the mean square error between the value prediction clamp value and the time difference target is calculated as the Critic loss function.

[0219] Based on the loss function, the gradient of the Critic network parameters in the current iteration is calculated . Before updating the parameters using the gradient, the AGS mechanism is intervened and the gradient obtained by each backpropagation is dynamically constrained to ensure stability. In the adaptive gradient stabilization mechanism (AGS), the two-norm of the gradient is calculated :

[0220] ;

[0221] wherein represents the square value of the th element in the gradient, denotes the summation function.

[0222] Set the maximum gradient threshold , when the gradient exceeds the threshold, the gradient is scaled in proportion to keep it in the stable interval, and the scaled gradient is obtained. Finally, according to the scaled gradient, the parameters of the Critic network are updated by the optimizer (such as Adam). The formula for proportional scaling is:

[0223] ;

[0224] wherein denotes the scaled gradient, which limits the maximum step size of a single update, effectively avoiding gradient explosion; denotes a stability term to prevent division by zero.

[0225] After updating the parameters of the Critic network, the parameters of the Actor network are updated. Based on the state vector of the current iteration , generate actions by the current Actor network , and evaluate their values by the updated Critic network , calculate the Actor loss function of the Actor network in order to maximize the value. The Actor loss function adopts the loss function commonly used by the Actor network, which is not described in detail.

[0226] According to the Actor loss function, the gradient of the Actor network parameter is calculated, and the AGS mechanism is used to process the gradient of the Actor network parameter, that is, the process of updating the Critic network parameter is the same, and the updated Actor network parameter is obtained.

[0227] In addition, in the lightweight Actor / Critic network of the embodiment, the structure for value / policy mapping in the first MLP layer and the second MLP layer adopts a linear layer and a layer normalization structure to enhance the forward stability, so as to ensure that the gradient is not amplified in the propagation process. The formula corresponding to the linear layer and the layer normalization structure is:

[0228] ;

[0229] wherein, , respectively represent the weight and bias term of the linear layer and the layer normalization structure, represent the key features, represent the normalization, represent the output data of the key feature channel.

[0230] In the embodiment, AGS provides a stable and safe adjustment direction for network parameter update by scaling the excessively large gradient; Q-Clamp provides a reasonable and reliable target benchmark for training by limiting the output range of the value network. The two work together: Q-Clamp ensures that the loss calculation is based on valid values, thereby producing the correct gradient direction; AGS then amplitude-constrains the gradient to prevent oscillation or divergence in the update process. This joint mechanism fundamentally guarantees the training stability and convergence reliability of the lightweight DRL model in a complex dynamic environment.

[0231] S4-4, update the task queue state corresponding to the communication task and the computing task by using the same method as S1-5;

[0232] S4-5, repeat S4-1 to S4-4, and use the updated D3PG-Light model for iteration until the stable convergence condition is met, to generate a final resource allocation scheme.

[0233] In summary, the present application has the following significant technical effects by constructing a unified dynamic environment model, designing a lightweight D3PG-Light network structure, introducing LSTM time series modeling, innovating the feature fusion module IFF, the adaptive gradient stabilization mechanism AGS, and the segmented multi-objective reward function:

[0234] (1) The cross-module collaborative optimization of communication, sensing and computing resources is realized. The traditional method independently schedules each module, leading to resource conflicts and waste. The present application can dynamically allocate bandwidth, sensing and computing resources according to channel state, queue load and sensing demand through a unified resource model and joint action space, significantly improving resource utilization efficiency and reducing delay jitter.

[0235] (2) Strong time series robustness and training stability. The LSTM module effectively captures the time series changes of the channel and queue, combined with the AGS gradient stabilization mechanism, making the system have fast adaptive and recovery ability to channel mutations, task bursts and queue surges. The training process converges faster and has less oscillation, with a performance retention rate of more than 85% in unknown scenarios, overcoming the limitations of traditional DRL "simulation effective, real ineffective".

[0236] (3) Achieve extreme model lightweight and real-time deployment. Through the IFF module, single-layer LSTM and lightweight MLP design, the DRL parameter quantity is compressed from about 500,000 to about 15,000, reducing by 97%. On embedded platforms such as Jetson Orin Nano, single inference time ≤10ms, memory usage reduced by more than 70%, energy consumption reduced by 40%-60%, supporting 10-20Hz real-time decision frequency, and for the first time realizing online operation of DRL joint scheduling strategy on unmanned aerial vehicle lightweight devices.

[0237] (4) Design an efficient multi-objective reward mechanism. The segmented reward function optimizes communication throughput, sensing accuracy, computing completion rate, energy consumption and system long-term stability, has a strong punishment mechanism for queue overflow and extreme delay, is more sensitive to sensing accuracy improvement, and tends to global optimization when resources are scarce, significantly improving strategy quality and overall system performance.

[0238] As shown in Figure 4 , a UAV ISCC joint resource scheduling system based on lightweight DRL includes:

[0239] A vector generation module for collecting UAV multi-dimensional scheduling data, tasks, task queue states and perception-computation feedback of the previous time slot in the current time slot to generate a state vector of the current time slot; the UAV multi-dimensional scheduling data includes communication data and real A2G link measurement data of the UAV; the tasks include perception tasks, computing tasks and communication tasks;

[0240] The construction module is configured to construct the D3PG-Light model based on an IFF feature fusion mechanism, an LSTM network, and an MLP.

[0241] The scheme initial generation module is configured to process the state vector by the D3PG-Light model to generate an initial resource allocation scheme and a state value of a current time slot.

[0242] The scheme final generation module is configured to execute the initial resource allocation scheme by the UAV, calculate a task delay and a multi-target reward, update the D3PG-Light model and perform iteration based on the multi-target reward, and generate a final resource allocation scheme, wherein the task delay includes a communication delay, a calculation delay, and a perception delay.

[0243] It should be noted that, as to the system in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0244] Embodiment 2

[0245] In order to verify the embodiment 1, the original D3PG baseline (G0), the D3PG-Light model in the embodiment 1 (G1), the DDPG model (G2), the TD3 model (G3), the D3PG-Light model in the embodiment 1 without the adaptive gradient stabilization mechanism (G1-AGS), the D3PG-Light model without the IFF feature fusion module (G1-noIFF), the D3PG-Light model without the LSTM network (G1-noLSTM), and the D3PG-Light model without the Q-Clamp stabilization strategy (G1-noQ-Clamp) are taken for experiments, and the same experimental data are processed.

[0246] The training conditions are shown in Table 1.

[0247] Table 1

[0248]

[0249] The comparative experiment conditions are shown in Table 2.

[0250] Table 2

[0251]

[0252] The final average reward of G1 is close to G0, G2 and G3, and the average reward is 40±1. In terms of model parameter quantity, compared with the original G2, G1 is reduced by 97%, and compared with G2 and G3, G1 is also reduced by an order of magnitude. This shows that through network lightweight and algorithm improvement, although the model complexity is greatly reduced, G1 (the application) can still achieve the same income as the large model.

[0253] In terms of learning speed, the number of training rounds required for convergence of each algorithm is not statistically different: G1 reaches the convergence criterion at about the 942th round on average, which is basically the same as the 943rd round of G2 and the 944th round of the original G0, and G3 is slightly later. This shows that G1 does not sacrifice the training convergence speed to exchange for model compression, on the contrary, its convergence performance is as good as the existing algorithm.

[0254] Each algorithm also shows similar levels in task accuracy and resource utilization. Among them, the average perceptual accuracy of G1 is about 75.6%, which is slightly lower than the 78.5% of G2 and G3, but still remains at a high level. This slight gap may be due to G1 more actively exploring resource allocation at the beginning of training, but overall it can achieve close accuracy.

[0255] The average resource utilization rate of G1 is 98.3%, which is significantly higher than other baseline algorithms. This shows that the strategy generated by G1 tends to make full use of available resources to improve performance.

[0256] The time consumption of each step of reasoning of G0 is about 0.12ms, and the time consumption of DDPG is about 0.125ms, while the average reasoning time consumption of G1 is only about 0.49ms, which is still within milliseconds, although the LSTM module is introduced.

[0257] For G1, G1-AGS, G1-noIFF, G1-noLSTM and G1-noQ-Clamp, G1-noQ-Clamp has the greatest damage to performance, with a final average reward of -35.5, which is much lower than the positive value of G1, and the return variance is as high as 102.9, and the training appears serious instability and divergence. This result shows that the modeling of time series state is very important in the UAV MEC scene.

[0258] The influence of G1-noIFF is relatively small. The final average reward of G1-noIFF is about 37.2, which is about 2 points lower than the complete model, and the IFF feature fusion module may become an important source of gain in more complex or multi-task scenarios.

[0259] The adaptive gradient stabilization mechanism (AGS) has a significant impact on the algorithm performance. After removing AGS, the average reward of the agent in G1-AGS drops to 24.3, which is about 38% lower than that of the complete algorithm. The instability in the training process increases, and the number of extreme negative reward rounds rises to 61.2, which is almost one-third higher than the baseline. Some training curves show oscillation or even a sharp drop in reward.

[0260] The effect of the Q-Clamp stabilization strategy cannot be ignored. After removing Q-clamp, the average reward of G1-noQ-Clamp drops to 32.1, which is about 18% lower than that of the complete algorithm. The number of extreme negative reward rounds increases to 58.8. The results prove that the algorithm performance fluctuates more severely without Q-value clipping: critics are more likely to produce inaccurate extreme estimates, which in turn interferes with the update of the actor policy, leading to instability in training and performance degradation. Q-clamp can be considered as a stabilizing means that adds luster to the flower and plays a role in ensuring the rationality of the value network output.

[0261] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

[0262] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application. The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A UAV ISCC joint resource scheduling method based on lightweight DRL, characterized in that, include: Collect UAV multidimensional scheduling data, tasks, task queue status, and perception-computation feedback from the previous time slot in the current time slot to generate the state vector of the current time slot; the UAV multidimensional scheduling data includes UAV communication data and real A2G link measurement data; the tasks include perception tasks, computation tasks, and communication tasks; A D3PG-Light model is constructed based on the IFF feature fusion mechanism, LSTM network, and MLP. The state vector is processed using the D3PG-Light model to generate the initial resource allocation scheme and state value for the current time slot; The initial resource allocation scheme is executed through UAV simulation, and task latency and multi-objective rewards are calculated. Based on the multi-objective rewards, the D3PG-Light model is updated and iterated to generate the final resource allocation scheme. The task delay includes communication delay, computation delay, and perception delay; The generation of the final resource allocation scheme includes: The initial resource allocation scheme is executed by UAV simulation. Based on the communication rate of the current time slot, the computing task, and the task queue status corresponding to the communication task, the communication latency, computing latency, sensing latency, and sensing accuracy indicators are calculated. Based on communication latency, computation latency, perception latency, and perception accuracy metrics, a multi-objective reward is calculated; the multi-objective reward includes total latency cost, perception accuracy reward, throughput efficiency reward, resource consumption penalty, energy consumption penalty, cascading extreme state penalty, and action smoothing penalty. Based on the multi-objective reward and state value of the current time slot, the D3PG-Light model is updated through an adaptive gradient stabilization mechanism and Q-value clamping. Update the task queue status corresponding to the communication task and the computing task; iterate using the updated D3PG-Light model until the stable convergence condition is met, and generate the final resource allocation scheme. The communication delay includes inherent communication delay and queuing delay, and the corresponding formula is: ； ； ； in, , , These represent inherent communication delay, queuing delay, and communication delay, respectively. Indicates the size of the data packets processed by the communication task. This represents a small constant to prevent division by zero. Indicates communication rate. This indicates the task queue status corresponding to the communication task in the current time slot; The computation delay includes inherent computation delay and queuing delay, and the corresponding formula is: ； ； ； ； in, , , These represent inherent computation delay, queuing delay, and computation delay, respectively. This represents a small constant to prevent division by zero. Indicates the computing power of MEC. Indicates the size of a single computational task. This indicates the proportion of resources allocated to the computing task in the current time slot. express, This indicates the task queue status corresponding to the computation task in the current time slot; The formula corresponding to the sensing delay is: ； in, This indicates the proportion of sensing resources required by the sensing task in the current time slot. Represents the SNR gain factor. Indicates perceived latency. Indicates the reference sensing time constant. Indicates the signal-to-noise ratio; The formula corresponding to the sensing accuracy index is: ； in, Indicates the perception accuracy index. This represents an exponential function with base e. , These represent the shape parameters of the control curve slope and the threshold, respectively.

2. The UAV ISCC joint resource scheduling method based on lightweight DRL according to claim 1, characterized in that, The process of generating the state vector for the current time slot includes: Collect UAV multidimensional scheduling data for the current time slot, set the time slot length and ISCC parameters, and build the ISCC environment; Channel calculation data is obtained by using the sliding window length to calculate the channel gain sequence in real A2G link measurement data; Based on real A2G link measurement data, the signal-to-noise ratio and communication rate are calculated using the clip function and Shannon's formula. The communication distance and flight altitude of the UAV are determined based on the location information of the UAV, and the statistical characteristics of the A2G channel are generated based on the channel calculation data, signal-to-noise ratio and communication rate. Based on the amount of communication data and computation tasks of the UAV in the current time slot, update the task queue status corresponding to the communication tasks and computation tasks. Obtain the perception-computation feedback from the previous time slot and the UAV tasks in the current time slot; construct the state vector of the current time slot based on the updated task queue state and A2G height statistical features.

3. The UAV ISCC joint resource scheduling method based on lightweight DRL according to claim 1, characterized in that, The D3PG-Light model includes a cascaded IFF feature fusion module, an LSTM module, an Actor network, and a Critic network; both the Actor network and the Critic network include a first MLP layer and a second MLP layer. The IFF feature fusion module includes a channel feature branch, a queue feature branch, a feature splicing layer, a third lightweight linear layer, and a third activation function layer; the channel feature branch includes a first lightweight linear layer and a first activation function layer; the queue feature branch includes a second lightweight linear layer and a second activation function layer.

4. The UAV ISCC joint resource scheduling method based on lightweight DRL according to claim 3, characterized in that, The initial resource allocation scheme and state value for generating the current time slot include: The state vector is input into the IFF feature fusion module to generate the communication-queue fusion feature of the current time slot; The communication-queue fusion features are input into the LSTM module for time-series evolution to generate the channel-queue time-series features of the current time slot; The channel-queue timing characteristics of the current time slot are input into the Actor network to generate the resource allocation action for the current time slot, thus obtaining the initial resource allocation scheme. The initial resource allocation scheme and channel-queue timing characteristics are combined and input into the Critic network to calculate the state value of the current time slot.

5. The UAV ISCC joint resource scheduling method based on lightweight DRL according to claim 1, characterized in that, The metrics for calculating communication latency, computation latency, sensing latency, and sensing accuracy include: Calculate the communication latency based on the communication rate of the current time slot and the task queue status corresponding to the communication task; Calculate the computation latency based on the task size and task queue status of each computation task in the current time slot; Based on the baseline sensing time constant and the signal-to-noise ratio of the current time slot, the sensing delay is calculated using a function with marginal decreasing characteristics; A perception accuracy model is constructed using a sigmoid activation function. Based on the signal-to-noise ratio of the current time slot, the perception accuracy index is calculated using the perception accuracy model.

6. The UAV ISCC joint resource scheduling method based on lightweight DRL according to claim 1, characterized in that, The calculation of multi-objective rewards includes: Based on communication latency, computation latency, and perceived latency, the total latency cost is calculated using the log function. Based on the perception accuracy index, the perception accuracy reward is calculated using a monotonically increasing function. Based on the communication data volume of each communication task, the average communication data volume is calculated, and combined with the task service volume of the communication task, the throughput efficiency reward is calculated. Calculate the resource occupancy penalty based on the initial resource allocation scheme; The energy consumption penalty is calculated based on the transmit power, the amount of computational tasks performed by the UAV, and the resources occupied by the sensing system. Based on preset latency thresholds, resource consumption thresholds, and energy consumption thresholds, combined with the initial total latency cost, energy consumption penalty, and initial resource allocation scheme, the cascaded extreme state penalty is calculated. Calculate the action smoothing penalty based on the initial resource allocation scheme of the current time slot and the previous time slot; The multi-objective reward is calculated by weighted fusion based on total latency cost, perception accuracy reward, throughput efficiency reward, resource consumption penalty, energy consumption penalty, cascaded extreme state penalty, and action smoothness penalty.

7. A UAV ISCC joint resource scheduling system based on lightweight DRL, used to implement the UAV ISCC joint resource scheduling method based on lightweight DRL as described in any one of claims 1 to 6, characterized in that, include: The vector generation module is used to collect UAV multidimensional scheduling data, tasks, task queue status, and perception-computation feedback from the previous time slot in the current time slot, and generate the state vector of the current time slot; the UAV multidimensional scheduling data includes UAV communication data and real A2G link measurement data; the tasks include perception tasks, computation tasks, and communication tasks; The building block is used to construct the D3PG-Light model based on the IFF feature fusion mechanism, LSTM network and MLP; The initial scheme generation module is used to process the state vector through the D3PG-Light model to generate the initial resource allocation scheme and state value for the current time slot; The final solution generation module is used to simulate the execution of the initial resource allocation scheme through UAV, calculate task latency and multi-objective rewards; based on the multi-objective rewards, update the D3PG-Light model and iterate to generate the final resource allocation scheme; the task latency includes communication latency, computation latency and perception latency.

Images