Method and device for task scheduling and resource allocation of post-disaster unmanned aerial vehicle communication system

Abstract

The present application relates to the field of communication technology, in particular to a method and device for task scheduling and resource allocation of a post-disaster unmanned aerial vehicle (UAV) sensing system. The method comprises: UAVs supplementing failed base stations and monitoring potential sensing targets; taking a communication penalty index and a sensing penalty index as optimization objectives under the energy constraint of the UAVs; establishing a joint task scheduling and resource allocation model to minimize the weighted sum of a communication performance index and a sensing performance index; re-describing the optimization problem as a Markov decision process (MDP); constructing a soft actor-critic (SAC) algorithm to update the MDP; applying transfer learning to the SAC algorithm; and transferring the parameters of the trained model to the updated MDP when the availability of the base stations changes. Experimental results show that the SAC algorithm is superior to baseline algorithms in terms of communication and sensing performance and convergence speed. The SAC algorithm based on transfer learning exhibits faster convergence speed and better robustness when the availability of the base stations changes suddenly.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a task scheduling and resource allocation method and device of a post-disaster unmanned aerial vehicle sensing and communication system. BACKGROUND

[0002] Integrated Sensing and Communication (ISAC) promotes the sharing of wireless resources, physical infrastructure and devices between sensing and communication functions, reducing costs and improving spectrum, energy and device utilization. ISAC has been identified by International Mobile Telecommunications (IMT) 2030 as one of the six key application scenarios for future Sixth Generation Wireless Systems (6G). Among various ISAC application cases, Unmanned Aerial Vehicles (UAVs) assisted ISAC systems for emergency rescue have received great attention. Due to fast deployment, flexible maneuvering and robust line-of-sight (LoS) link, UAVs can be quickly and efficiently dispatched to emergency sites while providing network services for Communication Users (CUs) and sensing services for Sensing Targets (STs).

[0003] The prior art proposes a new ISAC mechanism based on a single UAV, and converts and solves the problem through a convex optimization method. Similarly, the prior art also studies how to optimize the flight trajectory and beamforming strategy of a single UAV through a convex method. However, this method is only applicable to small-scale scenarios with usually only one UAV and simplified optimization objectives.

[0004] To cope with larger-scale dynamic scenarios, some researchers have adopted Deep Reinforcement Learning (DRL) algorithms, which use deep reinforcement learning to optimize the trajectories of multiple UAVs to balance communication and sensing performance. However, these studies are directed at scenarios where there is stable and reliable communication between ground base stations, communication users and UAVs.

[0005] In emergency rescue activities, the UAV-assisted ISAC task scheduling and resource allocation face unprecedented uncertainty, especially in terms of base station availability, which is still an unexplored field. After a disaster, due to the damage of some base stations, communication coverage blind spots may occur. In addition, when the availability of base stations changes (for example, some base stations are repaired), this change needs to re-run the optimization or re-train the deep reinforcement learning model, which is both time-consuming and computationally expensive. SUMMARY

[0006] To solve the technical problems in the prior art that in emergency rescue activities, the UAV-assisted ISAC task scheduling and resource allocation face unprecedented uncertainty, after a disaster, due to the damage of some base stations, communication coverage blind spots may occur, and when the availability of base stations changes, it is both time-consuming and computationally expensive, the embodiments of the present application provide a post-disaster UAV sensing system task scheduling and resource allocation method and device. The technical solution is as follows:

[0007] On the one hand, a post-disaster UAV sensing system task scheduling and resource allocation method is provided, characterized in that the method comprises:

[0008] S1, deploying a UAV to supplement the failed base station, and simultaneously monitoring potential sensing targets through the deployed UAV;

[0009] S2, constructing a communication performance index of the UAV, following the penalty function method in constraint optimization, taking the communication penalty index as the optimization target under the energy constraint of the UAV;

[0010] Constructing a sensing performance index of the UAV, following the penalty function method in constraint optimization, taking the sensing penalty index as the optimization target under the energy constraint of the UAV;

[0011] S3, establishing a joint task scheduling and resource allocation model, minimizing the weighted sum of the communication performance index and the sensing performance index through the joint task scheduling and resource allocation model, and obtaining an optimization problem;

[0012] S4, re-describing the optimization problem as a Markov decision process;

[0013] S5, obtaining the re-described optimization problem; constructing and training a soft actor-critic algorithm SAC model to update the Markov decision process, applying transfer learning to the soft actor-critic algorithm SAC model; when the availability of base stations changes, transferring the parameters of the trained soft actor-critic algorithm SAC model to the updated Markov decision process, obtaining the optimal value of the optimization variable, and then performing task scheduling and resource allocation of the post-disaster UAV sensing system.

[0014] Optionally, in S1, the unmanned aerial vehicles are deployed to supplement the failed base stations, and the potential sensing targets are monitored by the deployed unmanned aerial vehicles, including:

[0015] I base stations in a predefined service area U are acquired; network coverage blanks in the area U are acquired;

[0016] K unmanned aerial vehicles are deployed in the network coverage blanks in the area U, and the K unmanned aerial vehicles simultaneously detect L potential sensing targets on the ground;

[0017] Wherein, the unmanned aerial vehicles and the base stations are both equipped with omnidirectional antennas to broadcast information to communication users;

[0018] In each time slot, all unmanned aerial vehicles first fly to a predetermined position and then hover to provide ISAC service.

[0019] Optionally, in S2, the communication performance indicators of the unmanned aerial vehicles are acquired, and the communication penalty index is taken as the optimization objective under the energy constraint of the unmanned aerial vehicles by following the penalty function method in constraint optimization, including:

[0020] The communication performance indicators of the unmanned aerial vehicles are constructed; the communication performance indicators of the unmanned aerial vehicles include: the communication penalty index, the signal-to-interference-and-noise ratio, and the data rate of the communication user CU m ;

[0021] When a communication user successfully receives communication in a given time slot, the communication penalty index of the communication user is reset to 0; otherwise, the communication penalty index is increased by 1 based on the historical value of the communication penalty index, until the upper limit F max is reached.

[0022] If the data rate reaches the required threshold , it is considered that the communication user successfully receives communication, and vice versa; The sum of the communication penalty indexes of all communication users is

[0023] Wherein, represents the communication penalty index of the communication user CU m in time slot n, represents the communication penalty index of the communication user CU m in time slot n-1, F max represents the maximum value of the communication penalty index, represents the minimum value of the data rate of the communication user CU m in the nth time slot; the overall communication penalty index in time slot n is the sum of the penalty indexes of all communication users.

[0024] Optionally, in S2, the sensing performance index of the UAV is acquired, a penalty function method in constraint optimization is followed, the sensing penalty index is taken as an optimization target under the energy constraint of the UAV, and the following steps are included:

[0025] The sensing target is acquired.

[0026] The sensing performance index of the UAV is constructed. The sensing performance index includes the sensing penalty index.

[0027] When the radar estimated information rate of a sensing target is greater than a set threshold , it indicates that sensing is successful, that is, the value of the sensing state is 1.

[0028] When a sensing target is successfully sensed in a given time slot, the sensing penalty index of the sensing target is reset to 0; otherwise, the communication penalty index is increased by 1 based on the historical value of the communication penalty index, until the upper limit F max is reached.

[0029] wherein, represents the radar estimated information rate of the sensing target S l in the time slot n; represents the threshold of the radar estimated information rate of the sensing target S l in the time slot n, represents the sensing state of the sensing target S l in the time slot n, represents the sensing penalty index of the sensing target S l in the time slot n, represents the sensing penalty index of the sensing target S l in the time slot n-1.

[0030] Optionally, in S3, a joint task scheduling and resource allocation model is established, and a weighted sum of the communication performance index and the sensing performance index is minimized through the joint task scheduling and resource allocation model, including:

[0031] The joint task scheduling and resource allocation model is established.

[0032] The optimization target is acquired. The optimization target is to minimize a weighted sum of the communication penalty index and the sensing penalty index .

[0033] The optimization target is shown in the following formula (1):

[0034]

[0035] ​wherein the optimization objective includes three decision variables: a set of transmit powers P UAV , a set of UAV trajectories u, and a set of association indicators Ω.

[0036] The constraints that the optimization problem needs to satisfy include: a radar pulse power constraint for the UAV, a boundary constraint for the UAV, and a movement distance constraint for the UAV in a time slot n, EU k,n ≤ EU max an energy constraint for the UAV, α k,j,n ∈ {0, 1} and an association indicator constraint, all k, j, n must satisfy the above constraints.

[0037] wherein, and denote the minimum and maximum transmit power, denotes the x-coordinate of the UAV UAV k in time slot n, denotes the x-coordinate of the UAV UAV k in time slot n-1, denotes the y-coordinate of the UAV UAV k in time slot n, denotes the y-coordinate of the UAV UAV k in time slot n-1, denotes the z-coordinate of the UAV UAV k in time slot n, denotes the z-coordinate of the UAV UAV k in time slot n-1, max max max denote the maximum movement distance of the UAV in a time slot n in x, y, and z coordinates, respectively, EU max denotes the maximum energy value of the UAV, α k,j,n denotes the association indicator whether the UAV UAV k serves a communication user CU j or not.

[0038] Optionally, in S4, the optimization problem is re-expressed as a Markov decision process, including:

[0039] The optimization problem is converted into a Markov decision process, denoted as a four-element tuple (S, A, P, R);

[0040] wherein the state space set S n ​​Position information of all base stations Position information of all UAVs Position information of all communication users Position information of all perception targets And energy consumption of all UAVs

[0041] Action set A in time slot n n Common (4+J) * K elements contain flight distances of all UAVs Transmit power And a set of association indicators

[0042] Policy P = P(S n+1 |S n ,A) represents the probability of transferring to a new state S n After taking action A in state S n+1 ;

[0043] In time slot n, the reward R n Is defined as -τ(Y C +Y S )+a, where τ is a proportional constant, and a is a bias constant. When the constraints in the optimization problem cannot be met, R n Is R n -b, where b is a penalty term. The present invention sets b to a sufficiently large value.

[0044] Optionally, in S5, the re-expressed optimization problem is obtained; a soft actor-critic algorithm SAC model is constructed and trained to update the Markov decision process, and transfer learning is applied to the soft actor-critic algorithm SAC model. When the base station availability changes, the parameters of the trained model are transferred to the updated Markov decision process to obtain the optimal value of the optimization variable. This is followed by post-disaster UAV inter-sensing system task scheduling and resource allocation, including:

[0045] Obtain the re-expressed optimization problem

[0046] Combined with the principle of maximum entropy reinforcement learning and the advantages of the actor-critic framework, a soft actor-critic algorithm SAC model is obtained, and the Markov decision process is updated through the soft actor-critic algorithm SAC model;

[0047] Transfer learning is used on the SAC model to obtain a SAC model based on transfer learning, wherein after multiple iterations of training, the source task trained parameters of the source task are In transfer learning, the parameters of the target task are initialized by the source task trained parameters of the source task;

[0048] After initialization, continue to update the parameters of the target task in transfer learning using the SAC model until the performance converges; obtain the optimal value of the optimization variable; wherein the optimization variable is a decision variable in the optimization objective;

[0049] By using the SAC model based on transfer learning to adapt to the change of the post-disaster base station, the task scheduling and resource allocation of the post-disaster UAV sensing system are completed.

[0050] On the other hand, a task scheduling and resource allocation device for a post-disaster UAV sensing system is provided, which is applied to the task scheduling and resource allocation method for the post-disaster UAV sensing system, and the device comprises:

[0051] A UAV deployment module is configured to deploy UAVs to supplement failed base stations and monitor potential sensing targets through the deployed UAVs;

[0052] A performance index confirmation module is configured to construct a communication performance index of the UAV, and follow a penalty function method in constraint optimization to take the communication penalty index as an optimization objective under the energy constraint of the UAV;

[0053] An awareness performance index of the UAV is constructed, and a penalty function method in constraint optimization is followed to take the awareness penalty index as an optimization objective under the energy constraint of the UAV;

[0054] An optimization problem module is configured to establish a joint task scheduling and resource allocation model, minimize the weighted sum of the communication performance index and the awareness performance index through the joint task scheduling and resource allocation model, and obtain an optimization problem;

[0055] An optimization problem conversion module is configured to restate the optimization problem as a Markov decision process;

[0056] A communication and awareness module is configured to obtain the re-stated optimization problem, construct and train a soft actor-critic algorithm SAC model to update the Markov decision process, apply transfer learning to the soft actor-critic algorithm SAC model, transfer the parameters of the trained model to the updated Markov decision process when the base station availability changes, and obtain the optimal value of the optimization variable to perform the task scheduling and resource allocation of the post-disaster UAV sensing system.

[0057] On the other hand, a task scheduling and resource allocation device for a post-disaster UAV sensing system is provided, which comprises a processor and a memory, wherein the memory has computer readable instructions stored thereon, and the computer readable instructions are executed by the processor to implement any one of the above methods for task scheduling and resource allocation of the post-disaster UAV sensing system.

[0058] In another aspect, a computer-readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loaded and executed by a processor to implement any one of the methods of task scheduling and resource allocation of the post-disaster unmanned aerial vehicle common sensing system described above.

[0059] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0060] The embodiments of the present application consider a new type of post-disaster unmanned aerial vehicle assisted ISAC system, in which the unmanned aerial vehicle is used to supplement the network capacity of the ground base station that loses service, while using its radio signal for sensing. The present application applies transfer learning and deep reinforcement learning to learn task scheduling and resource allocation strategies, which can quickly adapt to the uncertainty of the availability of the ground base station. Experimental results show that the algorithm is superior to existing algorithms in terms of communication and sensing performance and convergence speed. In addition, when the availability of the ground base station changes abruptly, the DRL based on transfer learning has faster convergence and better robustness. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0062] Figure 1 is a flowchart of a task scheduling and resource allocation method of a post-disaster unmanned aerial vehicle common sensing system provided by the embodiments of the present application;

[0063] Figure 2 is a model diagram of unmanned aerial vehicle deployment provided by the embodiments of the present application;

[0064] Figure 3 is a flowchart of joint task scheduling and resource allocation of an unmanned aerial vehicle assisted ISAC network based on transfer learning provided by the embodiments of the present application;

[0065] Figure 4 is an iteration condition diagram of an optimization target in a transfer learning based SAC algorithm provided by the embodiments of the present application;

[0066] Figure 5 is an iteration condition diagram of cumulative rewards in a transfer learning based SAC algorithm provided by the embodiments of the present application;

[0067] Figure 6 is an iteration condition diagram of an optimization target in a SAC algorithm and a transfer learning based SAC algorithm provided by the embodiments of the present application;

[0068] Figure 7 is a trajectory diagram of a UAV after 10000 iterations of the SAC algorithm provided by an embodiment of the present application before repairing the ground base station;

[0069] Figure 8 is a trajectory diagram of a UAV after 10000 iterations of the SAC algorithm based on transfer learning provided by an embodiment of the present application after repairing the ground base station;

[0070] Figure 9 is a task scheduling and resource allocation device block diagram of a post-disaster UAV sensing system provided by an embodiment of the present application;

[0071] Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0072] The technical solutions in the present application will be described below with reference to the drawings.

[0073] In the embodiments of the present application, the words such as "example", "for example" and the like are used to represent as an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0074] In the embodiments of the present application, sometimes the subscript such as W1 may be written in the form of non-subscript such as W1, and when the difference is not emphasized, the meanings expressed are consistent.

[0075] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.

[0076] The embodiments of the present application provide a task scheduling and resource allocation method of a post-disaster UAV sensing system, which can be implemented by a task scheduling and resource allocation device of the post-disaster UAV sensing system. The task scheduling and resource allocation device of the post-disaster UAV sensing system can be a terminal or a server. As shown in the flow chart of the task scheduling and resource allocation method of the post-disaster UAV sensing system, the processing flow of the method can include the following steps: Figure 1

[0077] S1, deploy a UAV to supplement the failed base station, and at the same time monitor potential sensing targets through the deployed UAV;

[0078] ​In one feasible implementation, in S1, drones are deployed to supplement failed base stations, and simultaneously, the deployed drones are used to monitor potential targets, including:

[0079] Get I base stations within a predefined service area U; get the network coverage gaps within area U;

[0080] K drones are deployed in the network coverage gaps within region U, and the K drones simultaneously detect L potential sensing targets on the ground.

[0081] Both the drones and the base stations are equipped with omnidirectional antennas to broadcast information to communication users;

[0082] Within each time slot, all drones first fly to their designated locations and then hover to provide ISAC services.

[0083] In one feasible implementation, ground base stations primarily provide ground-to-ground communication services to communication users. This invention considers an unmanned aerial vehicle (UAV)-assisted ISAC system for emergency rescue operations, particularly addressing scenarios where some ground base stations lose service due to damage or power outages caused by disasters, resulting in network coverage gaps. Within a predefined service area U, there are I base stations, denoted as BS1, BS2, ..., BS... I Each base station (BS) i The operating state is α i ∈{0,1}, where 0 represents out of service and 1 represents in service. To fill gaps in network coverage, K UAVs are deployed in area U to provide air-to-ground communication services to M single-antenna users. The K UAVs simultaneously detect L potential ground targets. The K UAVs are denoted as UAV1, UAV2, ..., UAV... K M single-antenna users are represented as CU1, CU2, ..., CU M Let L be the perceived targets, S1, S2, ..., S L .

[0084] Model such as Figure 2 As shown. Both the drone and the base station are equipped with omnidirectional antennas to broadcast information to communication users. This invention discretizes a flight cycle T into several binary equal time slots. Within each time slot, all drones first fly to a predetermined position and then hover to provide ISAC service. In time slot n, let BS... i UAV k CU m and S l The three-dimensional position vectors are respectively represented as and

[0085] Following the penalty function method in constrained optimization, the application converts the goal of maximizing fair and efficient communication and perception performance into minimizing respective penalty values. In order to ensure the QoS of wide-area communication, efficient perception, fairness and energy saving, the application proposes two new indicators, i.e. a communication penalty index and a perception penalty index, which are taken as optimization goals under the energy constraint of the UAV.

[0086] S2, a communication performance indicator of the UAV is constructed, and following the penalty function method in constrained optimization, the communication penalty index is taken as an optimization goal under the energy constraint of the UAV;

[0087] A perception performance indicator of the UAV is constructed, and following the penalty function method in constrained optimization, the perception penalty index is taken as an optimization goal under the energy constraint of the UAV.

[0088] In a feasible implementation manner, in S2, the communication performance indicator of the UAV is acquired, and following the penalty function method in constrained optimization, the communication penalty index is taken as an optimization goal under the energy constraint of the UAV, including:

[0089] The communication performance indicator of the UAV is constructed; the communication performance indicator of the UAV includes: the communication penalty index, the signal-to-interference-and-noise ratio and the data rate of a communication user CU m .

[0090] When a communication user successfully receives communication in a given time slot, the communication penalty index of the communication user is reset to 0; otherwise, the communication penalty index is increased by 1 on the basis of its historical value , until it reaches an upper limit F max .

[0091] If the data rate reaches a required threshold , it is considered that the communication user successfully receives communication, and vice versa. The sum of the communication penalty indexes of all communication users is denoted as .

[0092] Wherein, denotes the communication penalty index of the communication user CU m at time slot n, denotes the communication penalty index of the communication user CU m at time slot n-1, F max denotes the maximum value of the communication penalty index, denotes the minimum value of the data rate of the communication user CU m in the nth time slot; the overall communication penalty index in the time slot n is the sum of the penalty indexes of all communication users.

[0093] In one possible implementation, the communication penalty index ensures both the communication performance and the fairness among the communication users. When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index is increased by 1 based on its historical value until it reaches an upper limit F max . If the data rate reaches a required threshold , the communication user is considered to have successfully received the communication, and vice versa. The sum of the communication penalty indices of all the communication users. A smaller penalty index means a more fair distribution among the communication users and a wider communication range. denotes the communication penalty index of the communication user CU m at time slot n, denotes the communication penalty index of the communication user CU m at time slot n-1, F max denotes the maximum value of the communication penalty index, denotes the minimum value of the data rate of the communication user CU m in the nth time slot. The overall communication penalty index in time slot n is the sum of the penalty indices of all the communication users.

[0094] The second indicator is the signal-to-interference-plus-noise ratio (SINR). The present application sets the base station and the UAV to use different subcarriers to transmit signals, so the signals of the base station and the UAV do not interfere with each other. The detailed SINR calculation process is in [Yaxi, Liu, Wei Huangfu, Huan Zhou, Haijun Zhang, Jiangchuan Liu, and Keping Long. Fair and energy-efficient coverage optimization for UAV placement problem in the cellular network. IEEE Trans. Commun., 70(6):4222-4235, June 2022.].

[0095] The third indicator is the data rate of the communication user CU m . In time slot n, the present application defines event E1 m,n as the maximum data rate m received by the communication user CU from the ground base station being greater than or equal to a predetermined threshold denotes the maximum data rate received by the communication user CUm Minimum value of data rate received from ground base station in time slot n. Communication User, CU m Maximum data rate received from ground base station in time slot n, i.e. Communication User, CU m Data rate received from ground base station in time slot n, mathematically expressed as Blog2(1 + SINR i,m,n ). B represents bandwidth. SINR i,m,n Communication User, CU m Signal to interference and noise ratio of communication. Data rate provided by UAV , mathematically expressed as:

[0096]

[0097] where I m,n represents interference term of Communication User, CU m in time slot n, mathematically expressed as Transmit power of UAV k in time slot n. Communication channel of UAV k to Communication User, CU m in time slot n, mathematically expressed as:

[0098]

[0099] where g1represents transmit antenna gain, g2represents receive antenna gain, λ CU represents signal wavelength. Data rate of ground base station to Communication User satisfies requirement, i.e. user receives successful communication. In addition, UAV is used as temporary supplement, thus Communication User tends to be served by ground base station preferentially. 2 represents power of Gaussian white noise. Transmit power of UAV to Communication User, CU m . is an index representing served UAV, mathematically expressed as argmax k α k,m,n , where α k,m,n represents association indicator of whether Communication User, CU k is served by UAV m , i.e. α k,m,n ∈{0, 1}. Communication channel of UAV to Communication User, CU mThe communication channel. This invention defines event E2. m,n To indicate that there is exactly one drone in time slot n. For communication user CU m This condition means that a communication user can only be provided by one drone. The service will be provided without being served by multiple drones simultaneously. When E1 m,n When it is 1, the communication user CU m The data rate is When E2 m,n =1 and E1 m,n When it is 0, the communication user CU m The data rate is Otherwise, the data rate is 0.

[0100] In one feasible implementation, in step S2, the perception performance index of the UAV is obtained, and following the penalty function method in constrained optimization, the perception penalty index is used as the optimization objective under the energy constraint of the UAV, including:

[0101] Acquire the perceived target;

[0102] Construct perception performance indicators for drones; perception performance indicators include: perception penalty index;

[0103] When the radar estimates the information rate Greater than the set threshold When this occurs, it indicates successful perception, i.e., a state of perception. The value is 1;

[0104] When a target is successfully perceived within a given time slot, its perception penalty index is... Reset to 0; otherwise, the communication penalty index. In its historical value Add 1 to the base value until the upper limit F is reached. max ;

[0105] in, This indicates that target S is sensed within time slot n. l Radar estimation information rate; This indicates that target S is sensed within time slot n. l The threshold for radar to estimate information rate. This indicates that target S is sensed within time slot n. l The state of perception, This indicates that target S is sensed within time slot n. l Perceived punishment index, This indicates that target S is sensed within time slot n-1. l The perceived punishment index.

[0106] In a feasible implementation, the sensing penalty index ensures both sensing performance and fairness among sensing targets. When the radar estimated information rate is greater than a set threshold , it indicates that sensing is successful, i.e., the value of the sensing state is 1. When a sensing target is successfully sensed in a given time slot, its sensing penalty index is reset to 0; otherwise, the communication penalty index is increased by 1 based on its historical value , until it reaches an upper limit F max . is the sum of the sensing penalty indices of all sensing targets. denotes the radar estimated information rate of the sensing target S l in time slot n, denotes the sensing state of the sensing target S l in time slot n, denotes the sensing penalty index of the sensing target S l in time slot n, denotes the sensing penalty index of the sensing target S l in time slot n-1, and F max denotes the maximum value of the sensing penalty index. denotes the radar estimated information rate of the sensing target S l in time slot n, which has a mathematical expression as follows:

[0107]

[0108] wherein, denotes the round-trip channel between the UAV U k and the sensing target S l , which is related to the transmitting antenna gain g3, the receiving antenna gain g4, the radar cross section σ r , and the wavelength λ S of the echo signal, the position of the UAV , and the position of the sensing target , and has a mathematical expression as follows:

[0109] In a feasible implementation, the energy consumption EU k of the UAV U k,n mainly includes communication energy consumption and propulsion energy consumption, EU k,nThe sum of both. The communication energy consumption includes communication circuit, signal processing, signal transmission / reception, etc. The propulsion energy consumption is used to support the UAV flight or hovering in the air. Here, the energy consumption model of the UAV adopted by the present application is the energy consumption in[Yaxi Liu, Wei Huangfu, Huan Zhou, Haijun Zhang, Jiangchuan Liu, and Keping Long. Fair and energy-efficient coverage optimization for UAV placement problem in the cellular network. IEEE Trans. Commun., 70(6):4222-4235, June 2022.]。

[0110] S3, a joint task scheduling and resource allocation model is established, and a weighted sum of the communication performance index and the perception performance index is minimized through the joint task scheduling and resource allocation model to obtain an optimization problem;

[0111] In an available implementation, in S3, the joint task scheduling and resource allocation model is established, and the weighted sum of the communication performance index and the perception performance index is minimized through the joint task scheduling and resource allocation model, including:

[0112] The joint task scheduling and resource allocation model is established;

[0113] An optimization target is obtained, and the optimization target is to minimize a weighted sum of a communication penalty index and a perception penalty index ;

[0114] The optimization target is shown in the following formula (1):

[0115]

[0116] Among them, the optimization target includes three decision variables: a set of transmission power P UAV , a set of UAV trajectories u, and a set of association indexes Ω;

[0117] The constraint conditions that the optimization problem needs to satisfy include: is the radar pulse power constraint of the UAV, is the boundary constraint of the UAV, and is the moving distance constraint of the UAV in a time slot n, EU k,n ≤ EU max is the energy constraint of the UAV, α k,j,n ∈{0, 1} and For the association indicator constraint, all k, j, n must satisfy the above constraint;

[0118] wherein, and denote the minimum and maximum transmit power, denote the x-coordinate of the UAV k in time slot n, denote the x-coordinate of the UAV k in time slot n-1, denote the y-coordinate of the UAV k in time slot n, denote the y-coordinate of the UAV k in time slot n-1, denote the z-coordinate of the UAV k in time slot n, denote the z-coordinate of the UAV k in time slot n-1, x max , y max , z max denote the maximum moving distance of the UAV in x, y, z coordinate in a time slot n, EU max denote the maximum energy value of the UAV, a k,j,n denote the association indicator whether a communication user CU k is served by the UAV j .

[0119] In one possible implementation, in order to ensure the communication and perception performance under the UAV energy consumption constraint, the present application aims to minimize the weighted sum of the communication penalty index and the perception penalty index , i.e. the mathematical expression of the optimization objective is The penalty index proposed in the present application ensures fair and effective perception, and fair and guaranteed QoS communication.

[0120] There are three decision variables in the optimization problem: the set of transmit powers the set of UAV trajectories the set of association indicators A certain communication user has already been served by the base station, and the communication user will not be assigned to any UAV. Therefore, the present application removes the elements related to the communication user already served by the base station from Ω. The present application sets J to represent the number of communication users served by the UAV. Ω can be rewritten as

[0121] The optimization problem needs to satisfy eight constraint conditions: the radar pulse power constraint of the UAV, a boundary constraint for the UAV, and a movement distance constraint for the UAV in a time slot n, EU k,n ≤ EU max an energy constraint for the UAV, a k,j,n ∈{0,1} and a relevance indicator constraint, all k,j,n must satisfy the above constraints. and denote the minimum and maximum transmit power, denote the x-coordinate of the UAV UAV k in time slot n, denote the x-coordinate of the UAV UAV k in time slot n-1, denote the y-coordinate of the UAV UAV k in time slot n, denote the y-coordinate of the UAV UAV k in time slot n-1, denote the z-coordinate of the UAV UAV k in time slot n, denote the z-coordinate of the UAV UAV k in time slot n-1, max max max x, y, z respectively denote the maximum movement distance of the UAV in a time slot n in the x-coordinate, y-coordinate, z-coordinate, EU max denote the maximum energy value of the UAV, a k,j,n denote the relevance indicator of whether the UAV UAV k serves a communication user CU j .

[0122] S4, restate the optimization problem as a Markov decision process;

[0123] In one possible implementation, S4, restate the optimization problem as a Markov decision process, including:

[0124] In order to perform deep reinforcement learning and improve scalability, the present application converts the optimization problem into a Markov decision process, denoted as a four-element tuple (S, A, P, R). The present application will introduce the basic elements of deep reinforcement learning and Markov decision process:

[0125] Agent: The remote base station serves as a centralized agent accessible to all UAVs, responsible for decision-making for all variables.

[0126] ​​Environment: The environment is the UAV-assisted ISAC system for emergency rescue activities.

[0127] State (S): The state space set S at time slot n n including the location information of all base stations including the location information of all UAVs including the location information of all communication users including the location information of all perception targets and the energy consumption of all UAVs

[0128] Action (A): The action set A at time slot n n There are (4+J) x K elements in total, including the flight distance of all UAVs transmit power and the set of association indicators

[0129] Policy (P): P = P (S n+1 |S n ,A) represents the probability of transferring to a new state S n after taking action A in state S n+1 .

[0130] Reward (R): In time slot n, the reward R n is defined as -τ (Y C + Y S ) + a, where τ is a proportional constant and a is a bias constant. When the constraints in the optimization problem cannot be met, R n is R n -b, where b is a penalty term. In this invention, b is set to a sufficiently large value.

[0131] The goal of DRL is usually to maximize the cumulative reward, so solving the above Markov decision process can ultimately solve the optimization problem of this patent.

[0132] S5, obtain the re-expressed optimization problem; build and train the soft actor-critic algorithm SAC model to update the Markov decision process, apply transfer learning to the soft actor-critic algorithm SAC model, transfer the parameters of the trained model to the updated Markov decision process when the base station availability changes, and obtain the optimal value of the optimization variable. This is followed by the task scheduling and resource allocation of the post-disaster UAV inter-sensing system.

[0133] In a feasible implementation manner, in S5, the re-expressed optimization problem is obtained; a soft actor-critic algorithm SAC is used to update a Markov decision process, and transfer learning is applied to the soft actor-critic algorithm SAC, and when the base station availability changes, the parameters of the trained model are transferred to the updated Markov decision process, including:

[0134] the re-expressed optimization problem is obtained

[0135] In combination with the principle of maximum entropy reinforcement learning and the advantages of the actor-critic framework, a soft actor-critic algorithm SAC is obtained, and the soft actor-critic algorithm SAC is used to update the Markov decision process;

[0136] Transfer learning is used on the SAC algorithm to obtain a SAC based on transfer learning, wherein after multiple iterations of training, the parameters of the source task trained by the source task are In transfer learning, the parameters of the target task are initialized by the parameters of the source task trained by the source task;

[0137] After initialization, the parameters of the target task in transfer learning are continuously updated using the SAC algorithm until the performance converges, and the optimal value of the optimization variable is obtained; wherein the optimization variable is a decision variable in the optimization target;

[0138] By using the SAC algorithm based on transfer learning, the task scheduling and resource allocation of the post-disaster unmanned aerial sensing system are completed by adapting to the changes of the post-disaster base station.

[0139] In a feasible implementation manner, the application uses the most advanced DRL algorithm, namely the SAC algorithm, to solve the Markov decision process. The SAC algorithm is specially designed for continuous action space, and combines the principle of maximum entropy reinforcement learning and the advantages of the actor-critic framework, and provides an algorithm that can achieve efficient learning in a complex environment. There are five neural networks in the SAC algorithm, including a policy network π(A|S), two Q value networks and two target Q value networks and θ1 represents the parameters of the first Q value network, Ω2 represents the parameters of the second Q value network, θ1' represents the parameters of the first target Q value network, and θ2' represents the parameters of the second target Q value network.

[0140] Firstly, the parameters of the neural network are initialized, and the parameters of the policy network π(A|S), the Q value networks and and the target Q value networks and parameters and initializing a temperature parameter, epsilon, that controls the strength of entropy regularization. Further, the present application interacts with the environment, collects a number of state transitions (S n n n n+1 ) and stores them in an experience replay buffer. S n represents a set of state spaces at time slot n, A n represents a set of actions at time slot n, R n represents a reward at time slot n, S n+1 represents a set of state spaces at time slot n+1. Next, the present application updates the network parameters using a batch of state transitions sampled at random. The implementation steps of the SAC algorithm are as follows:

[0141] 1) Update the Q-value networks

[0142] The present application updates the two Q-value networks by minimizing (1) and (2), and the two Q-value network loss functions are:

[0143]

[0144] L Q (θ1) represents the loss function of the first Q-value network, L Q (θ2) represents the loss function of the second Q-value network. y is the target value, and the calculation expression is:

[0145]

[0146] where S' represents the new state, gamma represents the discount factor, epsilon represents the temperature parameter, and the action is π(·|S').

[0147] The Q-value network parameters θ1 and θ2 are updated according to the following rules:

[0148]

[0149]

[0150] where eta represents the learning rate, represents the gradient of L Q (θ1), represents the gradient of L Q (θ2).

[0151] 2) Update the policy network

[0152] The parameters of the policy network are updated by maximizing the weighted sum of the expected return and the entropy of the policy, and the loss function is:

[0153] ​​​

[0154] Policy network parameters The policy network parameters can be updated by the following way:

[0155]

[0156] 3) Soft update target Q-value network

[0157] The update expression is:

[0158]

[0159] 4) Update ε

[0160] Adjust ε adaptively by minimizing the loss:

[0161]

[0162] wherein, represents the target policy entropy.

[0163] 5) Repeat step

[0164] Steps 1)-4) are repeated continuously until the algorithm converges.

[0165] In a feasible implementation, when some ground base stations are repaired, part of the communication users originally served by the unmanned aerial vehicle will be served by the newly repaired base stations, so that the number of communication users served by the unmanned aerial vehicle is reduced. Subsequently, in the time slot n, the action space set A n and the reward function R n will be adjusted according to different numbers of communication users. For the sake of clarity, the present application represents the updated Markov decision process as The present application will use transfer learning on the SAC algorithm and be called transfer learning-based SAC. By using the transfer learning-based SAC algorithm to quickly adapt to the changes of post-disaster base stations, emergency ISAC services are provided for the system.

[0166] After multiple iterations of training, the parameters of the source task trained are In transfer learning, the parameters of the target task are initialized by these parameters of the source task, that is:

[0167]

[0168] wherein, represents the parameters in the trained policy network in the source task, represents the parameters in the trained first Q-value network in the source task, represents the parameters in the trained second Q-value network in the source task, This represents the parameters of the first target Q-value network trained in the source task. This represents the parameters of the second objective Q-value network trained in the source task. θ represents the parameters of the policy network in transfer learning. 1,target θ represents the parameters of the first Q-value network in transfer learning. 2,target θ′ represents the parameters of the second Q-value network in transfer learning. 1,target θ′ represents the parameters of the Q-value network for the first objective in transfer learning. 2,target This represents the parameters of the Q-value network for the second objective in transfer learning. This represents the first Q-value network in transfer learning. This represents the second Q-value network in transfer learning. This represents the Q-value network as the first objective in transfer learning. This represents the Q-value network representing the second objective in transfer learning. This represents the first Q-value network in the source task. This represents the second Q-value network in the source task. This represents the Q-value network of the first objective in the source task. This represents the Q-value network of the second objective in the source task.

[0169] In one feasible implementation, the SAC algorithm is used to continue updating the parameters of the target task in transfer learning until performance converges. It is important to note that the source and target tasks have different numbers of elements in their action spaces. To address this issue, this invention... Several zero elements were added to make This allows parameters from the source task to be directly passed to parameters in the target task. The algorithm flow is as follows: Figure 3 As shown.

[0170] In one feasible implementation, a disaster scenario is modeled, and a simulated scenario is modeled using synthetic data. The effectiveness of the algorithm proposed in this invention is then discussed.

[0171] First, consider a small-scale disaster scenario. In a rectangular area R∈[-1000m, 1000m]×[-1200m, 1200m], there are two drones, 17 ground base stations, 20 communication users (15 users are served by ground base stations and are represented by purple dots, while 5 users are not served by ground base stations and are represented by blue dots), and 5 sensing targets. The base station antennas have an 8° tilt angle, an antenna height of 21 meters to 50 meters, and a transmit power of 24 dBm. Figure 2 As shown. As an example of a post-disaster scenario, the remaining 7 antennas provide communication, operating at a low power of 6dBm, such as... Figure 3As shown. Here, 5 communication users are served by ground base stations (purple dots), while the other 15 communication users are not served (blue dots). Two drones provide emergency ISAC services to support communication users and sensed targets in coverage blind spots. The bandwidth B is set to 1MHz, the communication-related transmit antenna gain g1 is 5dB, the communication-related receive antenna gain g2 is 15dB, and the signal wavelength λ... CU The range is 0.1m, the sensing-related transmit antenna gain g3 is 15dB, the sensing-related receive antenna gain g4 is 15dB, and the radar cross-section σ r 1m 2 and echo signal wavelength λ S The minimum and maximum transmit power are 0.35m. and Set the maximum movement distances to 0.01W, 0.1W, and x-coordinate, y-coordinate, and z-coordinate respectively. max y max z max The maximum energy values ​​(EU) of the drones are 50m, 50m, and 5m respectively. max The penalty index is 3KJ, with an upper limit F. max The value is 100, and the communication user CU is in the nth time slot. m Data rate threshold 6×10 5 , bit / s, sensing target S within time slot n l Threshold for radar estimated information rate 10 5 bit / s, discount factor γ is 0.98, learning rate η is 0.01, target policy entropy The value is 20, the proportionality constant τ is 1 / 50, the bias constant a is 1.8, the penalty term b is 20, and the noise power σ is... 2 10 -11 .

[0172] To evaluate the convergence and effectiveness of the proposed transfer learning-based SAC algorithm, this invention... Figure 4 The image shows the iterative process of optimizing the objective using the SAC algorithm based on transfer learning. Figure 4 As can be seen, with the increase of the number of iterations, the objective function value gradually decreases and eventually converges, indicating that the algorithm in this paper is effective.

[0173] from Figure 5As can be clearly seen in FIG. 6, the cumulative reward shows a gradual upward trend as the number of iterations increases. This indicates that during the training process, the agent continuously learns and optimizes its decision-making strategy, resulting in better performance in each iteration. The increase in cumulative reward reflects the agent's gradual learning of better decisions in the complex environment, effectively improving the success rate of communication and perception tasks and optimizing the utilization efficiency of resources. This cumulative reward growth trend verifies the effectiveness of the adopted algorithm and demonstrates the algorithm's continuous improvement and optimization capabilities in task allocation and resource scheduling in the UAV-assisted ISAC system.

[0174] Figure 6 FIG. 7 shows the optimization objective under the SAC algorithm and the transfer learning-based SAC algorithm. From Figure 6 As can be seen in FIG. 7, the optimization objective gradually decreases as the number of iterations increases. This trend indicates that both algorithms can effectively optimize task allocation and resource scheduling strategies, thereby reducing the overall optimization objective value. In contrast, the transfer learning-based SAC algorithm converges faster and converges to a better solution. This means that the algorithm has higher adaptability and efficiency in handling dynamic environmental changes. Overall, Figure 6 The results in FIG. 7 verify the effectiveness of these two algorithms in optimizing task scheduling and resource allocation in the UAV-assisted ISAC system.

[0175] To further demonstrate the performance of the algorithms, the present invention visualizes the trajectories of the UAVs after using the SAC algorithm for 10,000 iterations before repairing the ground base station. Figure 7 In FIG. 8, the trajectories of the two UAVs are represented by red and green colors, respectively, with small solid circles of the corresponding color indicating the paths flown by the UAVs. Triangles represent the starting points of the UAVs, while pentagons represent the end points. From Figure 7 It can be seen that the UAVs fly almost over every communication user and perception target, providing ISAC dual functions. In addition, the UAVs rarely fly over areas lacking communication users and perception targets. Furthermore, the UAVs hover back and forth over communication users and perception targets to ensure fairness and QoS of communication and perception.

[0176] The present invention shows the trajectories of the UAVs generated by the transfer learning-based SAC algorithm after 10,000 iterations in Figure 8 From Figure 8 It can be seen that the UAVs successfully adjust their strategies to provide ISAC services for updated unserved communication users and perception targets, while avoiding serving communication users covered by the repaired ground base station. During the flight, the UAVs also provide ISAC services by hovering back and forth.

[0177] In summary, the SAC algorithm, especially the SAC algorithm based on transfer learning, demonstrates excellent performance in UAV-assisted ISAC networks, particularly in the case of uncertainty in base station availability, where it can effectively perform joint task scheduling and resource allocation.

[0178] This invention proposes a novel post-disaster UAV-assisted ISAC system, in which the availability of ground base stations dynamically changes. This invention deploys UAVs to supplement failed base stations while simultaneously monitoring potential sensing targets to understand the situation in the disaster area. The main contributions of this invention are as follows: First, to balance the relationship between communication and sensing, this invention proposes two novel indices: a communication penalty index and a sensing penalty index, to ensure wide-area communication, effective sensing, and fair quality of service (QoS). Under UAV energy constraints, this invention establishes a joint task scheduling and resource allocation model to minimize the weighted sum of these indices. Second, to reduce time complexity and improve scalability, this invention reformulates the optimization problem as a Markov decision process (MDP). The corresponding MDP adjusts as base station availability changes. Finally, this invention proposes an advanced deep reinforcement learning algorithm—the Soft Actor-Critic (SAC) algorithm—to solve the Markov decision process problem. To enhance robustness and shorten training time, this invention applies transfer learning to the SAC algorithm. When base station availability changes, the parameters of the trained model are transferred to the updated Markov decision process. Experimental results using base station locations and configurations show that SAC outperforms baseline algorithms in both communication and sensing performance, as well as convergence speed. Furthermore, the transfer learning-based SAC exhibits faster convergence and better robustness when base station availability changes abruptly.

[0179] Figure 9 This is a block diagram of a task scheduling and resource allocation device 300 for a post-disaster unmanned aerial vehicle (UAV) sensing system, according to an exemplary embodiment. The device 300 is used for a task scheduling and resource allocation method for a post-disaster UAV sensing system. (Refer to...) Figure 9 The device includes a drone deployment module 310, a performance indicator verification module 320, an optimization problem module 330, an optimization problem transformation module 340, and a communication and sensing module 350. Among them:

[0180] The drone deployment module 310 is used to deploy drones to supplement failed base stations, and at the same time monitor potential targets through the deployed drones;

[0181] The performance index confirmation module 320 is used to construct the communication performance index of the UAV, following the penalty function method in constrained optimization, and taking the communication penalty index as the optimization objective under the energy constraint of the UAV.

[0182] We construct perception performance indicators for UAVs, following the penalty function method in constrained optimization, and take the perception penalty index as the optimization objective under the energy constraints of UAVs.

[0183] The optimization problem module 330 is used to establish a joint task scheduling and resource allocation model. The optimization problem is obtained by minimizing the weighted sum of communication performance indicators and perception performance indicators through the joint task scheduling and resource allocation model.

[0184] The optimization problem transformation module 340 is used to reformulate the optimization problem as a Markov decision process.

[0185] The communication and sensing module 350 is used to obtain the restated optimization problem; construct and train the soft actor-critic algorithm SAC model to update the Markov decision process; apply transfer learning to the soft actor-critic algorithm SAC model; when the availability of the base station changes, transfer the parameters of the trained model to the updated Markov decision process to obtain the optimal value of the optimization variable; and then perform task scheduling and resource allocation for the post-disaster UAV sensing system.

[0186] Optionally, the UAV deployment module 310 is used to acquire I base stations within a predefined service area U; and to acquire network coverage gaps within the area U.

[0187] K drones are deployed in the network coverage gaps within region U, and the K drones simultaneously detect L potential sensing targets on the ground.

[0188] Both the drones and the base stations are equipped with omnidirectional antennas to broadcast information to communication users;

[0189] Within each time slot, all drones first fly to their designated locations and then hover to provide ISAC services.

[0190] Optionally, the performance indicator verification module 320 is used to construct the communication performance indicators of the UAV; the communication performance indicators of the UAV include: communication penalty index, signal-to-interference-plus-noise ratio, and communication user CU. m data rate

[0191] When a communication user successfully receives communication within a given time slot, its communication penalty index is... Reset to 0; otherwise, the communication penalty index. In its historical value Add 1 to the base value until the upper limit F is reached. max ;

[0192] if the data rate reaches a required threshold then the communication user is considered to receive the communication successfully, and vice versa; the sum of the communication penalty indices of all the communication users;

[0193] wherein, denotes the communication penalty index of the communication user CU m at time slot n, denotes the communication penalty index of the communication user CU m at time slot n-1, F max denotes the maximum value of the communication penalty index, denotes the minimum value of the data rate of the communication user CU m in the nth time slot; the overall communication penalty index in time slot n is the sum of the penalty indices of all the communication users.

[0194] Optionally, the performance index confirmation module 320 is configured to acquire the sensing target;

[0195] construct a sensing performance index of the UAV; the sensing performance index comprises a sensing penalty index;

[0196] when the radar estimated information rate of the sensing target is greater than a set threshold , it indicates that the sensing is successful, i.e., the value of the sensing state is 1;

[0197] when a sensing target is successfully sensed in a given time slot, the sensing penalty index of the sensing target is reset to 0; otherwise, the communication penalty index is increased by 1 based on its historical value until it reaches an upper limit F max ;

[0198] wherein, denotes the radar estimated information rate of the sensing target S l in time slot n; denotes the threshold of the radar estimated information rate of the sensing target S l in time slot n, denotes the sensing state of the sensing target S l in time slot n, denotes the sensing penalty index of the sensing target S l in time slot n, denotes the sensing penalty index of the sensing target S l in time slot n-1.

[0199] Optionally, the optimization problem module 330 is configured to construct a joint task scheduling and resource allocation model;

[0200] The optimization objective is to minimize the communication penalty exponent. and perceived punishment index The weighted sum;

[0201] The optimization objective is shown in the following formula (1):

[0202]

[0203] The optimization objective includes three decision variables: the transmit power set P. UAV The drone trajectory set u and the associated indicator set Ω;

[0204] The constraints that the optimization problem needs to satisfy include: Radar pulse power constraints for UAVs For drone boundary constraints, as well as EU is a constraint on the distance traveled by a UAV within a time slot n. k,n ≤EU max For the energy constraints of the drone, α k,j,n ∈{0,1} and For the association indicator constraint, all k, j, n must satisfy the above constraint;

[0205] in, and Indicates the minimum and maximum transmit power. This indicates the number of unmanned aerial vehicles (UAVs) within time slot n. k x-coordinate This indicates the number of unmanned aerial vehicles (UAVs) within time slot n-1. k x-coordinate This indicates the number of unmanned aerial vehicles (UAVs) within time slot n. k The ordinate, This indicates the number of unmanned aerial vehicles (UAVs) within time slot n-1. k The ordinate, This indicates the number of unmanned aerial vehicles (UAVs) within time slot n. k height, This indicates the number of unmanned aerial vehicles (UAVs) within time slot n-1. k Height, x max y max z max These represent the maximum x-coordinate movement distance, maximum y-coordinate movement distance, and maximum z-coordinate movement distance of the UAV within a time slot n, respectively. max α represents the maximum energy value of the drone. k,j,n Indicates whether it is a drone (UAV) k Serving communication users CUj The associated indicator.

[0206] Optionally, the optimization problem transformation module 340 is used to transform the optimization problem into a Markov decision process, represented as a four-element tuple (S,A,P,R);

[0207] Among them, the state space set S within time slot n n Includes location information of all base stations Location information of all drones Location information of all communication users Location information of all sensed targets And the energy consumption of all drones

[0208] Action set A within time slot n n There are (4+J)×K elements, including the flight distances of all drones. Transmit power and associated indicator set

[0209] Strategy P = P(S) n+1 |S n A) indicates that in state S n After taking action A, the state transitions to the new state S. n+1 The probability of;

[0210] Within time slot n, the reward is R. n Defined as -τ(Y) C +Y S R = τ + a, where τ is a proportionality constant and a is a bias constant. When the constraints in the optimization problem cannot be satisfied, R... n For R n -b, where b is a penalty term. In this invention, b is set to a sufficiently large value.

[0211] Optionally, the communication and sensing module 350 is used to acquire the restated optimization problem.

[0212] By combining the principles of maximum entropy reinforcement learning with the advantages of the actor-critic framework, a soft actor-critic algorithm (SAC) is obtained, which is then used to update the Markov decision process.

[0213] Transfer learning is applied to the SAC algorithm to obtain a transfer learning-based SAC. After multiple iterations of training, the parameters trained on the source task are... In transfer learning, the parameters of the target task are initialized using the parameters trained on the source task.

[0214] After initialization, the SAC algorithm is used to update the parameters of the target task in transfer learning until the performance converges.

[0215] By using the SAC algorithm based on transfer learning to adapt to the change of the post-disaster base station, the task scheduling and resource allocation of the post-disaster UAV ISAC system are completed.

[0216] In the embodiments of the present application, a new post-disaster UAV assisted ISAC system is considered, in which the UAV is used to supplement the network capacity of the ground base station that loses service, and at the same time uses its radio signal for sensing. The present application applies transfer learning and deep reinforcement learning to learn task scheduling and resource allocation strategies, which can quickly adapt to the uncertainty of the availability of ground base stations. Experimental results show that the algorithm is superior to existing algorithms in terms of communication and sensing performance and convergence speed. In addition, when the availability of the ground base station changes suddenly, the DRL based on transfer learning has faster convergence and better robustness.

[0217] Figure 10 is a structural schematic diagram of a task scheduling and resource allocation device of a post-disaster UAV ISAC system provided by the embodiments of the present application, as shown in Figure 10 The task scheduling and resource allocation device of the post-disaster UAV ISAC system can include the post-disaster UAV ISAC system task scheduling and resource allocation device shown in Figure 9 Optionally, the task scheduling and resource allocation device of the post-disaster UAV ISAC system 410 can include a first processor 2001.

[0218] Optionally, the task scheduling and resource allocation device of the post-disaster UAV ISAC system 410 can further include a memory 2002 and a transceiver 2003.

[0219] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for task scheduling and resource allocation of a post-disaster UAV sensing system, characterized in that, The method comprises: S1, deploying unmanned aerial vehicles to supplement the failed base stations, and monitoring potential sensing targets through the deployed unmanned aerial vehicles; S2, constructing a communication performance index of the unmanned aerial vehicles, following a penalty function method in constraint optimization, taking a communication penalty index as an optimization target under the energy constraint of the unmanned aerial vehicles; The communication performance indicators of the UAV include: a communication penalty index, a signal-to-interference-and-noise ratio, and a data rate of a communication user CU m ​ When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index is incremented by 1 based on its historical value until it reaches an upper limit F max ; if the data rate reaches a required threshold the communication user is considered to have received the communication successfully, and vice versa; wherein denotes the communication penalty index of the communication user CU m at time slot n, denotes the communication penalty index of the communication user CU m at time slot n-1, F max denotes the maximum value of the communication penalty index, denotes the minimum value of the data rate of the communication user CU m in the n-th time slot; constructing a sensing performance index of the unmanned aerial vehicles, following a penalty function method in constraint optimization, taking a sensing penalty index as an optimization target under the energy constraint of the unmanned aerial vehicles; acquiring sensing targets; constructing a sensing performance index of the unmanned aerial vehicles; the sensing performance index comprises a sensing penalty index; When the radar estimates the information rate is greater than a set threshold it indicates a successful perception, i.e. the value of the perception state is 1; When a sensing target is successfully sensed in a given time slot, its sensing penalty index is reset to 0; otherwise, the sensing penalty index is incremented by 1 based on its historical value until it reaches an upper limit F max’ ; wherein represents the radar estimation information rate of a perceived target S l in timeslot n; represents a threshold value for the radar estimation information rate of a perceived target S l in timeslot n, represents the perception state of a perceived target S l in timeslot n, represents the perception penalty index of a perceived target S l in timeslot n, represents the perception penalty index of a perceived target S l in timeslot n-1; S3, establishing a joint task scheduling and resource allocation model, minimizing a weighted sum of the communication performance index and the sensing performance index through the joint task scheduling and resource allocation model, and obtaining an optimization problem; S4, re-describing the optimization problem as a Markov decision process; S5, acquiring the re-described optimization problem; constructing and training a soft actor-critic algorithm SAC model to update the Markov decision process, applying transfer learning to the soft actor-critic algorithm SAC model; when the base station availability changes, transferring the parameters of the trained soft actor-critic algorithm SAC model to the updated Markov decision process, obtaining the optimal value of the optimization variable, and then performing task scheduling and resource allocation of the post-disaster unmanned aerial vehicle sensing system.

2. The method of claim 1, wherein, In the S1, the deployment of unmanned aerial vehicles to supplement the failed base stations, and the monitoring of potential sensing targets through the deployed unmanned aerial vehicles, comprises: acquiring I base stations in a predefined service area U; acquiring network coverage blanks in the area U; deploying K unmanned aerial vehicles in the network coverage blanks in the area U, and the K unmanned aerial vehicles simultaneously detecting L sensing targets on the ground; wherein the unmanned aerial vehicles and the base stations are both equipped with omnidirectional antennas to broadcast information to communication users; in each time slot, all the unmanned aerial vehicles first fly to a predetermined position, and then hover to provide ISAC services.

3. The method of claim 2, wherein, In the S3, the establishment of the joint task scheduling and resource allocation model to minimize the weighted sum of the communication performance index and the sensing performance index, comprises: establishing a joint task scheduling and resource allocation model; an optimization objective of minimizing a communication penalty index and a perception penalty index a weighted sum of; a sum of communication penalty indices of all communication users in slot n; a sum of perception penalty indices of all communication users in slot n; then the optimization target is shown in the following formula (1): Among them, the optimization target includes three decision variables: the set of transmission power The set of UAV trajectories The set of association indicators When a communication user has been served by a base station, the communication user is not assigned to any drone, so the elements related to the communication user that has been served by the base station are removed from Ω, and Ω is rewritten as The constraints that the optimization problem needs to satisfy include: a radar pulse power constraint for the UAV, a boundary constraint for the UAV, and a movement distance constraint for the UAV in a time slot n, EU k,n ≤ EU max an energy constraint for the UAV, a k,j,n ∈{0,1} and an association indicator constraint, all k, h, n must satisfy the above constraints; wherein, and denote the minimum and maximum transmission power, denote the x-coordinate of the UAV k in the time slot n, denote the x-coordinate of the UAV k in the time slot n-1, denote the y-coordinate of the UAV k in the time slot n, denote the y-coordinate of the UAV k in the time slot n-1, denote the z-coordinate of the UAV k in the time slot n, denote the z-coordinate of the UAV k in the time slot n-1, max max max denote the maximum moving distance of the UAV in a time slot n in x, y, z coordinates, respectively, EU max denote the maximum energy value of the UAV, a k,j,n denote the association indicator whether the communication user CU k is served by the UAV j or not.​​ 4. The method of claim 3, wherein, In the S4, the re-description of the optimization problem as a Markov decision process, comprises: converting the optimization problem into a Markov decision process, represented as a four-element tuple (S, A, P, R); wherein the state space set S n location information of all base stations location information of all drones location information of all communication users location information of all perception targets and energy consumption of all drones Action set A within time slot n n Common (4+J) x K elements, containing flight distances for all drones Transmit power And a set of association indicators Policy P = P(S n+1 |S n , A) represents the probability of transitioning to a new state S n after taking action A in state S n+1 ; In slot n, the reward R n is defined as -τ(Y C + Y S ) + a, where τ is a proportional constant, a is a bias constant, and R n is R n - b, where b is a penalty term, which is set to a sufficiently large number when the constraints in the optimization problem cannot be satisfied.

5. The method of claim 4, wherein, In the S5, the acquisition of the re-described optimization problem; the construction and training of a soft actor-critic algorithm SAC model to update the Markov decision process, the application of transfer learning to the soft actor-critic algorithm SAC model, the transfer of the parameters of the trained model to the updated Markov decision process when the base station availability changes, the obtaining of the optimal value of the optimization variable, and then the task scheduling and resource allocation of the post-disaster unmanned aerial vehicle sensing system, comprises: acquiring the reformulated optimization problem The soft actor-critic (SAC) model is obtained by combining the principle of maximum entropy reinforcement learning and the advantage of the actor-critic framework, and the SAC model is used to update the Markov decision process; In the SAC model, transfer learning is adopted to obtain an SAC model based on transfer learning, wherein, represents parameters of a first Q value network, represents parameters of a second Q value network, represents parameters of a first target Q value network, represents parameters of a second target Q value network, and represents parameters of a policy network After multiple iterations of training, the parameters of the source task trained are In the transfer learning, the parameters of the target task are initialized by the parameters of the source task trained. After initialization, the SAC model is used to update the parameters of the target task in transfer learning until the performance converges, and the optimal value of the optimization variable is obtained; the optimization variable is a decision variable in the optimization objective; The SAC model based on transfer learning is used to adapt to the change of the post-disaster base station, and the task scheduling and resource allocation of the post-disaster UAV sensing system are completed.

6. A device for task scheduling and resource allocation of a post-disaster UAV situational awareness system, the device being configured to implement the method for task scheduling and resource allocation of the post-disaster UAV situational awareness system according to any one of claims 1-5, wherein, The device comprises: A UAV deployment module for deploying UAVs to supplement failed base stations and monitoring potential sensing targets through the deployed UAVs; A performance index confirmation module for constructing the communication performance index of the UAV, following the penalty function method in constraint optimization, and taking the communication penalty index as the optimization objective under the energy constraint of the UAV; An awareness performance index of the UAV is constructed, following the penalty function method in constraint optimization, and taking the awareness penalty index as the optimization objective under the energy constraint of the UAV; An optimization problem module for establishing a joint task scheduling and resource allocation model to minimize the weighted sum of the communication performance index and the awareness performance index, and obtaining an optimization problem; An optimization problem conversion module for re-describing the optimization problem as a Markov decision process; A communication and awareness module for obtaining the re-described optimization problem, constructing and training a soft actor-critic (SAC) model to update the Markov decision process, applying transfer learning to the SAC model, and transferring the parameters of the trained SAC model to the updated Markov decision process when the availability of the base station changes, and obtaining the optimal value of the optimization variable to perform the task scheduling and resource allocation of the post-disaster UAV sensing system.

7. A task scheduling and resource allocation device for a post-disaster UAV sensing system, characterized in that, The task scheduling and resource allocation device of the post-disaster UAV sensing system comprises: A processor; A memory having computer readable instructions stored thereon, which, when executed by the processor, implement the method of any one of claims 1 to 5.

8. A computer readable storage medium, characterized in that, The computer readable storage medium stores program code that can be called and executed by the processor to implement the method of any one of claims 1 to 5.

Images