Task scheduling and resource allocation method and device for post-disaster unmanned aerial vehicle sensing system

Abstract

The invention relates to the technical field of communication, in particular to a task scheduling and resource allocation method and device for a post-disaster unmanned aerial vehicle sensing system. The method comprises the following steps: the unmanned aerial vehicle supplements a failed base station and monitors a potential sensing target; taking the communication penalty index and the perception penalty index as optimization targets under the energy constraint of the unmanned aerial vehicle; establishing a joint task scheduling and resource allocation model, and minimizing the weighted sum of the communication performance index and the perception performance index; the optimization problem is re-expressed as a Markov decision process; and constructing a soft actor-commentator algorithm SAC to update the MDP, applying transfer learning to the soft actor-commentator algorithm SAC, and transferring the parameters of the trained model to the updated MDP when the availability of the base station changes. Experimental results show that the SAC is superior to a baseline algorithm in communication and sensing performance and convergence speed. The SAC based on transfer learning shows higher convergence speed and better robustness when the availability of the base station changes suddenly.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a method and device for task scheduling and resource allocation of a post-disaster unmanned aerial vehicle telepathy system. Background Art

[0002] Integrated Sensing and Communication (ISAC) promotes the sharing of wireless resources, physical infrastructure and devices between sensing and communication functions, reducing costs and improving the utilization of spectrum, energy and equipment. ISAC has been identified by International Mobile Telecommunications (IMT) 2030 as one of the six key application scenarios of the future Sixth Generation Wireless Systems (6G). Among various ISAC application cases, unmanned aerial vehicles (UAVs) assisted ISAC systems for emergency rescue have received great attention. Due to rapid deployment, flexible maneuverability and robust line-of-sight links, UAVs can be quickly and efficiently dispatched to emergency locations, while providing network services for communication users (CUs) and sensing services for sensing targets (STs).

[0003] The prior art proposes a new ISAC mechanism based on a single UAV and transforms and solves the problem through convex optimization methods. Similarly, the prior art also studies how to optimize the flight trajectory and beamforming strategy of a single UAV through convex methods. However, this method is only applicable to small-scale scenarios with only one UAV and simplified optimization objectives.

[0004] To cope with larger-scale dynamic scenarios, some researchers have adopted deep reinforcement learning (DRL) algorithms, which use deep reinforcement learning to optimize the trajectories of multiple drones to balance communication and perception performance. However, these studies are aimed at scenarios where there is stable and reliable communication between ground base stations, communication users, and drones.

[0005] In emergency rescue activities, drone-assisted ISAC task scheduling and resource allocation face unprecedented uncertainties, especially in terms of base station availability, which is still an underexplored area. After a disaster, communication coverage blind spots may occur due to the damage of some base stations. In addition, when the availability of base stations changes (for example, some base stations are repaired), this change requires re-running the optimization or retraining the deep reinforcement learning model, which is time-consuming and computationally expensive. Summary of the invention

[0006] In order to solve the technical problems that the drone-assisted ISAC task scheduling and resource allocation in emergency rescue activities in the prior art face unprecedented uncertainty, and that after a disaster, due to the damage of some base stations, communication coverage blind spots may occur, and when the availability of base stations changes, it is both time-consuming and computationally expensive, the embodiment of the present invention provides a task scheduling and resource allocation method and device for a post-disaster drone interawareness system. The technical solution is as follows:

[0007] On the one hand, a method for task scheduling and resource allocation of a post-disaster UAV synaesthesia system is provided, characterized in that the method comprises:

[0008] S1. Deploy drones to supplement failed base stations and monitor potential sensing targets through deployed drones;

[0009] S2. Construct the communication performance index of the UAV, follow the penalty function method in constrained optimization, and use the communication penalty index as the optimization target under the energy constraint of the UAV;

[0010] Construct the perception performance index of the UAV, follow the penalty function method in constrained optimization, and use the perception penalty index as the optimization target under the energy constraint of the UAV;

[0011] S3, establishing a joint task scheduling and resource allocation model, minimizing the weighted sum of the communication performance index and the perception performance index through the joint task scheduling and resource allocation model, and obtaining the optimization problem;

[0012] S4. Reformulate the optimization problem as a Markov decision process;

[0013] S5. Obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm (SAC) model to update the Markov decision process, and apply transfer learning to the soft actor-critic algorithm (SAC) model; when the base station availability changes, transfer the parameters of the trained soft actor-critic algorithm (SAC) model to the updated Markov decision process, obtain the optimal values ​​of the optimization variables, and then perform task scheduling and resource allocation for the post-disaster UAV synaesthesia system.

[0014] Optionally, in S1, a drone is deployed to supplement the failed base station, and the deployed drone is used to monitor potential sensing targets, including:

[0015] Obtain I base stations within a predefined service area U; obtain the blank portion of network coverage within area U;

[0016] Deploy K drones in the blank part of the network coverage in area U, and K drones simultaneously detect L potential sensing targets on the ground;

[0017] Among them, both drones and base stations are equipped with omnidirectional antennas to broadcast information to communication users;

[0018] In each time slot, all drones first fly to the predetermined location and then hover to provide ISAC services.

[0019] Optionally, in S2, the communication performance index of the UAV is obtained, and the penalty function method in the constrained optimization is followed to use the communication penalty index as the optimization target under the energy constraint of the UAV, including:

[0020] Construct the communication performance indicators of the UAV; the communication performance indicators of the UAV include: communication penalty index signal to noise ratio and communication user CU m Data rate;

[0021] When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0022] If the data rate reaches the required threshold It is considered that the communication user has received the communication successfully, and vice versa; is the sum of the communication penalty indexes of all communication users;

[0023] in, Indicates the communication user CU in time slot n m The communication penalty index, The communication user CU in time slot n-1 is represented as m The communication penalty index, F max represents the maximum value of the communication penalty index, Represented as the communication user CU in the nth time slot m Minimum value of data rate; overall communication penalty index within time slot n is the sum of the penalty indices of all communicating users.

[0024] Optionally, in S2, the perception performance index of the UAV is obtained, and the penalty function method in the constrained optimization is followed to use the perception penalty index as the optimization target under the energy constraint of the UAV, including:

[0025] Acquire perceived targets;

[0026] Construct the perception performance index of the UAV; the perception performance index includes: perception penalty index;

[0027] When the radar estimates the information rate Greater than the set threshold When , it indicates that the perception is successful, that is, the perception state The value of is 1;

[0028] When a perception target is successfully perceived within a given time slot, its perception penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0029] in, Indicates the target S is sensed in time slot n l The radar estimated information rate; Indicates the target S is sensed in time slot n l The threshold of radar estimated information rate, Indicates the target S is sensed in time slot n l The state of perception, Indicates the target S is sensed in time slot n l The perceived penalty index, Indicates that the target S is sensed in time slot n-1 l Perceived penalty index.

[0030] Optionally, in S3, a joint task scheduling and resource allocation model is established, and a weighted sum of a communication performance index and a perception performance index is minimized through the joint task scheduling and resource allocation model, including:

[0031] Establish joint task scheduling and resource allocation models;

[0032] Get the optimization target, which is to minimize the communication penalty index and perceived penalty index The weighted sum of

[0033] The optimization objective is as shown in the following formula (1):

[0034]

[0035] Among them, the optimization objective includes three decision variables: the transmission power set P UAV , UAV trajectory set u and associated indicator set Ω;

[0036] The constraints that need to be satisfied in the optimization problem include: is the radar pulse power constraint of the UAV, For the drone boundary constraints, as well as is the moving distance constraint of the UAV in a time slot n, EU k,n ≤EU max is the energy constraint of the UAV, α k,j,n ∈{0,1} and is the association indicator constraint, all k, j, n must satisfy the above constraints;

[0037] in, and Indicates the minimum and maximum transmit power, Indicates the number of UAVs in time slot n k The horizontal axis of Indicates that UAV in time slot n-1 k The horizontal axis of Indicates the number of UAVs in time slot n k The vertical coordinate of Indicates that UAV in time slot n-1 k The vertical coordinate of Indicates the number of UAVs in time slot n k Height, Indicates that UAV in time slot n-1 k The height, x max ,y max 、z max are the maximum moving distance of the x-coordinate, y-coordinate, and z-coordinate of the drone in a time slot n, respectively. max Represents the maximum energy value of the drone, α k,j,n Indicates whether it is a UAV k Serving communication users CU j The associated indicator.

[0038] Optionally, in S4, the optimization problem is reformulated as a Markov decision process, including:

[0039] Convert the optimization problem into a Markov decision process, represented as a four-element tuple (S, A, P, R);

[0040] Among them, the state space set S in time slot n nIncludes location information of all base stations Location information of all drones Location information of all communicating users Location information of all perceived targets And the energy consumption of all drones

[0041] The action set A in time slot n n There are a total of (4+J)×K elements, including the flight distances of all drones Transmit power and associated indicator sets

[0042] Strategy P = P(S n+1 |S n ,A) means in state S n After taking action A, it moves to the new state S n+1 The probability of

[0043] In time slot n, the reward R n Defined as -τ(Y C +Y S )+a, where τ is the proportional constant and a is the bias constant. When the constraints in the optimization problem cannot be met, R n For R n -b, where b is a penalty term. The present invention sets b to a sufficiently large value.

[0044] Optionally, in S5, a reformulated optimization problem is obtained; a soft actor-critic algorithm (SAC) model is constructed and trained to update the Markov decision process, transfer learning is applied to the soft actor-critic algorithm (SAC) model, and when the base station availability changes, the parameters of the trained model are transferred to the updated Markov decision process to obtain the optimal value of the optimization variable. Task scheduling and resource allocation of the post-disaster UAV synaesthesia system are then performed, including:

[0045] Get the reformulated optimization problem

[0046] Combining the principle of maximum entropy reinforcement learning with the advantages of the actor-critic framework, a soft actor-critic algorithm SAC model is obtained, and the Markov decision process is updated through the soft actor-critic algorithm SAC model;

[0047] Transfer learning is used on the SAC model to obtain a SAC model based on transfer learning. After multiple iterations of training, the trained parameters of the source task are In transfer learning, the parameters of the target task are initialized using the trained parameters of the source task;

[0048] After initialization, continue to use the SAC model to update the parameters of the target task in transfer learning until the performance converges; obtain the optimal value of the optimization variable; where the optimization variable is the decision variable in the optimization objective;

[0049] By using the SAC model based on transfer learning to adapt to the changes of base stations after the disaster, the task scheduling and resource allocation of the post-disaster UAV interawareness system are completed.

[0050] On the other hand, a task scheduling and resource allocation device for a post-disaster UAV synaesthesia system is provided, and the device is applied to a task scheduling and resource allocation method for a post-disaster UAV synaesthesia system, and the device comprises:

[0051] The drone deployment module is used to deploy drones to supplement failed base stations and monitor potential sensing targets through the deployed drones;

[0052] The performance index confirmation module is used to construct the communication performance index of the UAV, following the penalty function method in constrained optimization, and taking the communication penalty index as the optimization target under the energy constraint of the UAV;

[0053] Construct the perception performance index of the UAV, follow the penalty function method in constrained optimization, and use the perception penalty index as the optimization target under the energy constraint of the UAV;

[0054] An optimization problem module is used to establish a joint task scheduling and resource allocation model, and to obtain an optimization problem by minimizing the weighted sum of the communication performance index and the perception performance index through the joint task scheduling and resource allocation model;

[0055] Optimization problem transformation module, used to reformulate the optimization problem into a Markov decision process;

[0056] The communication and perception module is used to obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm (SAC) model to update the Markov decision process, apply transfer learning to the soft actor-critic algorithm (SAC) model, and transfer the parameters of the trained model to the updated Markov decision process when the base station availability changes to obtain the optimal value of the optimization variable. This is then used to perform task scheduling and resource allocation for the post-disaster UAV telepathy system.

[0057] On the other hand, a task scheduling and resource allocation device for a post-disaster drone synesthesia system is provided, and the task scheduling and resource allocation device for the post-disaster drone synesthesia system comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the task scheduling and resource allocation methods for the post-disaster drone synesthesia system is implemented.

[0058] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned task scheduling and resource allocation methods of the post-disaster drone synaesthesia system.

[0059] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0060] A novel post-disaster drone-assisted ISAC system is considered in the embodiments of the present invention, in which drones are used to supplement the network capacity of ground base stations that have lost service, while using their radio signals for perception. The present invention applies transfer learning and deep reinforcement learning to learn task scheduling and resource allocation strategies that can quickly adapt to uncertainties in the availability of ground base stations. Experimental results show that the algorithm outperforms existing algorithms in terms of communication and perception performance, as well as convergence speed. In addition, when the availability of ground base stations changes suddenly, DRL based on transfer learning has faster convergence and better robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 This is a flow chart of a task scheduling and resource allocation method for a post-disaster UAV synaesthesia system provided by an embodiment of the present invention;

[0063] Figure 2 is a diagram of a drone deployment model provided by an embodiment of the present invention;

[0064] Figure 3 It is a flow chart of UAV-assisted ISAC network joint task scheduling and resource allocation based on transfer learning provided by an embodiment of the present invention;

[0065] Figure 4 It is a diagram of the iteration of the optimization target provided by the embodiment of the present invention based on the transfer learning SAC algorithm;

[0066] Figure 5 It is a diagram of the iterative situation of the cumulative reward in the SAC algorithm based on transfer learning provided by an embodiment of the present invention;

[0067] Figure 6 It is a diagram of the iteration of the optimization target provided by the embodiment of the present invention under the SAC algorithm and the SAC algorithm based on transfer learning;

[0068] Figure 7 It is a trajectory diagram of the UAV after the SAC algorithm is iterated 10,000 times before repairing the ground base station provided by an embodiment of the present invention;

[0069] Figure 8 It is a trajectory diagram of the UAV after the SAC algorithm based on transfer learning is used to iterate 10,000 times after the ground base station is repaired according to an embodiment of the present invention;

[0070] Fig. 9 It is a block diagram of a task scheduling and resource allocation device for a post-disaster UAV interawareness system provided by an embodiment of the present invention;

[0071] Fig.10 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0073] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0074] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0075] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0076] The embodiment of the present invention provides a method for task scheduling and resource allocation of a post-disaster UAV interoception system, which can be implemented by a task scheduling and resource allocation device of the post-disaster UAV interoception system, and the task scheduling and resource allocation device of the post-disaster UAV interoception system can be a terminal or a server. Figure 1 The task scheduling and resource allocation method flow chart of the post-disaster UAV synaesthesia system is shown in the figure. The processing flow of the method may include the following steps:

[0077] S1. Deploy drones to supplement failed base stations and monitor potential sensing targets through deployed drones;

[0078] In a feasible implementation, in S1, a drone is deployed to supplement the failed base station, and at the same time, the deployed drone is used to monitor the potential sensing target, including:

[0079] Obtain I base stations within a predefined service area U; obtain the blank portion of network coverage within area U;

[0080] Deploy K drones in the blank part of the network coverage in area U, and K drones simultaneously detect L potential sensing targets on the ground;

[0081] Among them, both drones and base stations are equipped with omnidirectional antennas to broadcast information to communication users;

[0082] In each time slot, all drones first fly to the predetermined location and then hover to provide ISAC services.

[0083] In a feasible implementation, the ground base station mainly provides ground-to-ground communication services for communication users. The present invention considers a drone-assisted ISAC system for emergency rescue operations, especially for scenarios where some ground base stations lose service due to damage or power outages due to disasters, resulting in network coverage gaps. There are I base stations in the predefined service area U, denoted as BS1, BS2, ..., BS I , where each base station BS i The running state is α i ∈{0,1}, 0 means out of service, 1 means in service. To fill the gap in network coverage, K drones are deployed in area U to provide air-to-ground communication services for M single-antenna users. K drones simultaneously detect L potential sensing targets on the ground. K drones are denoted as UAV1, UAV2, …, UAV K , M single-antenna users are represented as CU1, CU2, …, CU M , L perception targets are represented as S1,S2,…,S L .

[0084] Models such as Figure 2 As shown. Both the drone and the base station are equipped with omnidirectional antennas to broadcast information to communication users. The present invention discretizes a flight cycle T into several binary equal time periods. In each time slot, all drones first fly to the predetermined position and then hover to provide ISAC services. In time slot n, let BS i 、UAV k , CU m and S l The three-dimensional position vectors are expressed as and

[0085] Following the penalty function method in constrained optimization, the present invention transforms the goal of maximizing fair and effective communication and perception performance into minimizing their respective penalty values. In order to ensure QoS of wide-area communication, effective perception, fairness and energy saving, the present invention proposes two new indicators, namely, communication penalty index and perception penalty index, and takes these two indicators as optimization goals under the energy constraint of UAV.

[0086] S2. Construct the communication performance index of the UAV, follow the penalty function method in constrained optimization, and use the communication penalty index as the optimization target under the energy constraint of the UAV;

[0087] The perception performance index of the UAV is constructed, and the penalty function method in constrained optimization is followed, and the perception penalty index is used as the optimization target under the energy constraint of the UAV.

[0088] In a feasible implementation, in S2, the communication performance index of the UAV is obtained, and the penalty function method in the constraint optimization is followed to use the communication penalty index as the optimization target under the energy constraint of the UAV, including:

[0089] Construct the communication performance indicators of the UAV; the communication performance indicators of the UAV include: communication penalty index signal to noise ratio and communication user CU m Data rate

[0090] When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0091] If the data rate reaches the required threshold It is considered that the communication user has received the communication successfully, and vice versa; is the sum of the communication penalty indexes of all communication users;

[0092] in, Indicates the communication user CU in time slot n m The communication penalty index, The communication user CU in time slot n-1 is represented as m The communication penalty index, F max represents the maximum value of the communication penalty index, Represented as the communication user CU in the nth time slot m Minimum value of data rate; overall communication penalty index within time slot n is the sum of the penalty indices of all communicating users.

[0093] In a feasible implementation, the communication penalty index ensures both communication performance and fairness among communication users. When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max If the data rate reaches the required threshold It is considered that the communication user has received the communication successfully, and vice versa. is the sum of the communication penalty indexes of all communication users. A smaller penalty index means more fairness among communication users and a wider communication range. Indicates the communication user CU in time slot n m The communication penalty index, The communication user CU in time slot n-1 is represented as m The communication penalty index, F max represents the maximum value of the communication penalty index, Represented as the communication user CU in the nth time slot m The minimum value of the data rate. The overall communication penalty index in time slot n is the sum of the penalty indices of all communicating users.

[0094] The second indicator is the signal-to-interference-plus-noise ratio (SINR). The present invention sets the base station and the UAV to use different subcarriers to transmit signals, so the signals of the base station and the UAV will not interfere with each other. The detailed signal-to-noise ratio calculation process is in [Yaxi, Liu, Wei Huangfu, Huan Zhou, Haijun Zhang, Jiangchuan Liu, and Keping Long. Fair and energy-efficient coverage optimization for UAV placement problem in the cellular network. IEEE Trans. Commun., 70(6): 4222–4235, June 2022.].

[0095] The third indicator is the communication user CU m In time slot n, the present invention defines event E1 m,n For communication users CU m Maximum data rate received from ground base station Greater than or equal to a predetermined threshold Indicates that the communication user CUm The minimum value of the data rate received from the ground base station in time slot n. Represented as communication user CU m The maximum data rate received from the ground base station in time slot n is Represented as communication user CU m The data rate received from the ground base station in time slot n is expressed as Blog2(1+SINR i,m,n ). B represents bandwidth. SINR i,m,n It is represented as the i-th ground base station to the communication user CU m The signal-to-interference-to-noise ratio of the communication. By drone The data rate provided is expressed mathematically as:

[0096]

[0097] Among them, I m,n Indicates the number of users CU communicating in time slot n m The mathematical expression of the interference term is Represented as UAV in time slot n k of transmission power. Represented as the UAV in time slot n k For communication users CU m The communication channel is expressed as:

[0098]

[0099] Where g1 represents the transmit antenna gain, g2 represents the receive antenna gain, and λ CU Expressed as signal wavelength. When the data rate from the ground base station to the communication user meets the requirements, it can be considered that the user has received successful communication. In addition, drones are used as temporary supplements, so communication users tend to be served by ground base stations first. 2 Expressed as the power of Gaussian white noise. Represented as a drone To the communication user CU m of transmission power. To represent the index of the served drone, the mathematical expression is argmax k α k,m,n , where α k,m,n Indicates whether it is operated by a UAV k Serving communication users CU m The association indicator, α k,m,n ∈{0,1}. Indicates drone For communication users CU mThe present invention defines event E2 m,n To indicate that there is exactly one drone in time slot n For communication users CU m This condition means that the communication user can only be represented by one drone service, rather than being served by multiple drones at the same time. m,n When it is 1, the communication user CU m The data rate is When E2 m,n is 1 and E1 m,n When it is 0, the communication user CU m The data rate is Otherwise, the data rate is 0.

[0100] In a feasible implementation, in S2, the perception performance index of the UAV is obtained, and the penalty function method in constrained optimization is followed to use the perception penalty index as the optimization target under the energy constraint of the UAV, including:

[0101] Acquire perceived targets;

[0102] Construct the perception performance index of the UAV; the perception performance index includes: perception penalty index;

[0103] When the radar estimates the information rate Greater than the set threshold , indicating that the perception is successful, that is, the perception state The value of is 1;

[0104] When a perception target is successfully perceived within a given time slot, its perception penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0105] in, Indicates the target S is sensed in time slot n l The radar estimated information rate; Indicates the target S is sensed in time slot n l The threshold of radar estimated information rate, Indicates the target S is sensed in time slot n l The state of perception, Indicates the target S is sensed in time slot n l The perceived penalty index, Indicates that the target S is sensed in time slot n-1 l The perceived penalty index.

[0106] In a feasible implementation, the perception penalty index ensures both the perception performance and the fairness between the perceived targets. Greater than the set threshold , indicating that the perception is successful, that is, the perception state The value of is 1. When a sensing target is successfully sensed within a given time slot, its sensing penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max . It is the sum of the perception penalty indices of all perception targets. Indicates the target S is sensed in time slot n l The threshold of radar estimated information rate, Indicates the target S is sensed in time slot n l The state of perception, Indicates the target S is sensed in time slot n l The perceived penalty index, Indicates that the target S is sensed in time slot n-1 l Perceived penalty index, F max Indicates the maximum value of the perceived penalty index. Indicates the target S is sensed in time slot n l The radar estimated information rate is expressed as:

[0107]

[0108] in, UAV k and the perceived target S l The round-trip channel is related to the transmit antenna gain g3, the receive antenna gain g4, and the radar cross section σ r and the echo signal wavelength λ S , the location of the drone and the location of the perceived target The mathematical expression is

[0109] In one feasible implementation, the UAV k Energy consumption EU k,n Mainly includes communication energy consumption and propulsion energy consumption, EU k,nis the sum of the two. Communication energy consumption includes communication circuits, signal processing, signal transmission / reception, etc. Propulsion energy consumption is used to support the UAV to fly or hover in the air. Here, the UAV energy consumption model adopted by the present invention is the energy consumption in [YaxiLiu, Wei Huangfu, Huan Zhou, Haijun Zhang, Jiangchuan Liu, and Keping Long. Fair and energy-efficient coverage optimization for UAV placement problem in the cellular network. IEEE Trans. Commun., 70(6): 4222–4235, June 2022.].

[0110] S3, establishing a joint task scheduling and resource allocation model, minimizing the weighted sum of the communication performance index and the perception performance index through the joint task scheduling and resource allocation model, and obtaining the optimization problem;

[0111] In a feasible implementation, in S3, a joint task scheduling and resource allocation model is established, and a weighted sum of a communication performance index and a perception performance index is minimized through the joint task scheduling and resource allocation model, including:

[0112] Establish joint task scheduling and resource allocation models;

[0113] Get the optimization target, which is to minimize the communication penalty index and perceived penalty index The weighted sum of

[0114] The optimization objective is as shown in the following formula (1):

[0115]

[0116] Among them, the optimization objective includes three decision variables: the transmission power set P UAV , UAV trajectory set u and associated indicator set Ω;

[0117] The constraints that need to be satisfied in the optimization problem include: is the radar pulse power constraint of the UAV, For the drone boundary constraints, as well as is the moving distance constraint of the UAV in a time slot n, EU k,n ≤EU max is the energy constraint of the UAV, α k,j,n ∈{0,1} and is the association indicator constraint, all k, j, n must satisfy the above constraints;

[0118] in, and Indicates the minimum and maximum transmit power, Indicates the number of UAVs in time slot n k The horizontal axis of Indicates that UAV in time slot n-1 k The horizontal axis of Indicates the number of UAVs in time slot n k The vertical coordinate of Indicates that UAV in time slot n-1 k The vertical coordinate of Indicates the number of UAVs in time slot n k Height, Indicates that UAV in time slot n-1 k The height, x max ,y max 、z max are the maximum moving distance of the x-coordinate, y-coordinate, and z-coordinate of the drone in a time slot n, respectively. max Represents the maximum energy value of the drone, α k,j,n Indicates whether it is a UAV k Serving communication users CU j The associated indicator.

[0119] In a feasible implementation, in order to ensure communication and perception performance under the energy consumption constraint of the UAV, the present invention aims to minimize the communication penalty index and perceived penalty index The weighted sum of , that is, the mathematical expression of the optimization objective is The penalty index proposed in the present invention ensures fair and effective sensing, as well as fair and QoS-guaranteed communication.

[0120] There are three decision variables in the optimization problem: the transmit power set Drone track collection Related indicator sets If a communication user is already served by a base station, then the communication user will not be assigned to any drone. Therefore, the present invention removes the elements related to the communication users already served by the base station from Ω. The present invention sets J to represent the number of communication users served by the drone. Ω can be rewritten as

[0121] The optimization problem needs to satisfy eight constraints: is the radar pulse power constraint of the UAV, For the drone boundary constraints, as well as is the moving distance constraint of the UAV in a time slot n, EU k,n ≤EU max is the energy constraint of the UAV, α k,j,n ∈{0,1} and is the association indicator constraint, all k, j, n must satisfy the above constraints. and Indicates the minimum and maximum transmit power, Indicates the number of UAVs in time slot n k The horizontal axis of Indicates that UAV in time slot n-1 k The horizontal axis of Indicates the number of UAVs in time slot n k The vertical coordinate of Indicates that UAV in time slot n-1 k The vertical coordinate of Indicates the number of UAVs in time slot n k Height, Indicates that UAV in time slot n-1 k The height, x max ,y max 、z max are the maximum moving distance of the x-coordinate, y-coordinate, and z-coordinate of the drone in a time slot n, respectively. max Represents the maximum energy value of the drone, α k,j,n Indicates whether it is a UAV k Serving communication users CU j The associated indicator.

[0122] S4. Reformulate the optimization problem as a Markov decision process;

[0123] In a feasible implementation, in S4, the optimization problem is reformulated as a Markov decision process, including:

[0124] In order to perform deep reinforcement learning and improve scalability, the present invention converts the optimization problem into a Markov decision process, represented as a four-element tuple (S, A, P, R). The present invention will introduce the basic elements of deep reinforcement learning and Markov decision process:

[0125] Agent: The remote base station acts as a centralized agent accessible to all drones and is responsible for making decisions on all variables.

[0126] Environment: The environment is a drone-assisted ISAC system used for emergency rescue activities.

[0127] State (S): The state space set S in time slot n n Includes location information of all base stations Location information of all drones Location information of all communicating users Location information of all perceived targets And the energy consumption of all drones

[0128] Action (A): Action set A in time slot n n There are a total of (4+J)×K elements, including the flight distances of all drones Transmit power and associated indicator sets

[0129] Policy (P): P = P (S n+1 |S n ,A) means in state S n After taking action A, it moves to the new state S n+1 probability.

[0130] Reward (R): In time slot n, reward R n Defined as -τ(Y C +Y S )+a, where τ is the proportional constant and a is the bias constant. When the constraints in the optimization problem cannot be met, R n For R n -b, where b is a penalty term. The present invention sets b to a sufficiently large value.

[0131] The goal of DRL is usually to maximize the cumulative reward, so solving the above Markov decision process can ultimately solve the optimization problem of this patent.

[0132] S5. Obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm (SAC) model to update the Markov decision process, apply transfer learning to the soft actor-critic algorithm (SAC) model, and transfer the parameters of the trained model to the updated Markov decision process when the base station availability changes, and obtain the optimal value of the optimization variable. This is then used to perform task scheduling and resource allocation for the post-disaster UAV synaesthesia system.

[0133] In one feasible implementation, in S5, a reformulated optimization problem is obtained; a soft actor-critic algorithm SAC is constructed to update a Markov decision process, transfer learning is applied to the soft actor-critic algorithm SAC, and when the base station availability changes, the parameters of the trained model are transferred to the updated Markov decision process, including:

[0134] Get the reformulated optimization problem

[0135] Combining the principle of maximum entropy reinforcement learning with the advantages of the actor-critic framework, the soft actor-critic algorithm SAC is obtained, and the Markov decision process is updated through the soft actor-critic algorithm SAC;

[0136] Transfer learning is used on the SAC algorithm to obtain SAC based on transfer learning. After multiple iterations of training, the trained parameters of the source task are In transfer learning, the parameters of the target task are initialized using the trained parameters of the source task;

[0137] After initialization, continue to use the SAC algorithm to update the parameters of the target task in transfer learning until the performance converges and the optimal value of the optimization variable is obtained; wherein the optimization variable is the decision variable in the optimization target;

[0138] By using the SAC algorithm based on transfer learning to adapt to the changes in base stations after the disaster, the task scheduling and resource allocation of the post-disaster UAV interawareness system are completed.

[0139] In a feasible implementation, the present invention uses the most advanced DRL algorithm, namely the SAC algorithm, to solve the Markov decision process. The SAC algorithm is specially designed for continuous action spaces and combines the principles of maximum entropy reinforcement learning and the advantages of the actor-critic framework to provide an algorithm that can achieve efficient learning in complex environments. There are 5 neural networks in the SAC algorithm, including a policy network π(A|S), two Q-value networks and Two target Q-value networks and θ1 represents the parameters of the first Q-value network, Ω2 represents the parameters of the second Q-value network, θ1′ represents the parameters of the first target Q-value network, and θ2′ represents the parameters of the second target Q-value network.

[0140] First, the present invention initializes the parameters of the neural network, initializing the strategy network π(A|S), the Q value network and Parameters, target Q value network and The parameters of the entropy regularization and the temperature parameter ε of the initialization control are used. Then, the present invention interacts with the environment and collects several state transitions (S n ,A n ,R n ,S n+1 ) and store it in the experience replay pool. n represents the state space set in time slot n, A n represents the action set in time slot n, R n represents the reward in time slot n, S n+1 Represents the state space set in time slot n+1. Next, the present invention uses a batch of randomly sampled state transitions to update network parameters. The implementation steps of the SAC algorithm are as follows:

[0141] 1) Update the Q value network

[0142] The present invention updates the two Q-value networks by minimizing (1) and (2), and the loss functions of the two Q-value networks are:

[0143]

[0144] L Q (θ1) represents the loss function of the first Q-value network, L Q (θ2) represents the loss function of the second Q-value network. y is the target value, and the calculation expression is:

[0145]

[0146] Among them, S′ represents the new state, γ represents the discount factor, ε represents the temperature parameter, and the action is π(·|S′).

[0147] The Q-value network parameters θ1 and θ2 are updated according to the following rules:

[0148]

[0149]

[0150] Among them, η represents the learning rate, Indicates L Q (θ1) finds the gradient, Indicates L Q (θ2) Find the gradient.

[0151] 2) Update the policy network

[0152] The parameters of the policy network are updated by maximizing the weighted sum of the expected return and entropy of the policy, and the loss function is:

[0153]

[0154] Policy network parameters It can be updated by:

[0155]

[0156] 3) Soft update target Q value network

[0157] Update the expression to:

[0158]

[0159] 4) Update ε

[0160] Adaptively adjust ε by minimizing the loss:

[0161]

[0162] in, represents the target policy entropy.

[0163] 5) Repeat steps

[0164] Repeat steps 1)-4) until the algorithm converges.

[0165] In a feasible implementation, when some ground base stations are repaired, some of the communication users originally served by the drone will be served by the newly repaired base stations, thereby reducing the number of communication users served by the drone. n and the reward function R n It will be adjusted according to the number of different communication users. For the sake of clarity, the present invention represents the updated Markov decision process as The present invention refers to the SAC algorithm that adopts transfer learning as SAC based on transfer learning. By using the SAC algorithm based on transfer learning to quickly adapt to the post-disaster base station changes, emergency ISAC services are provided for the system.

[0166] After multiple iterations of training, the trained parameters of the source task are In transfer learning, the parameters of the target task are initialized by these parameters of the source task, namely:

[0167]

[0168] in, represents the parameters of the policy network trained in the source task, represents the parameters of the first Q value network trained in the source task, represents the parameters of the second Q value network trained in the source task, represents the parameters of the first target Q value network trained in the source task, represents the parameters of the second target Q value network trained in the source task, represents the parameters of the policy network in transfer learning, θ 1,target represents the parameters of the first Q value network in transfer learning, θ 2,target represents the parameters of the second Q value network in transfer learning, θ′ 1,target Represents the parameters of the first target Q value network in transfer learning, θ′ 2,target Represents the parameters of the second target Q-value network in transfer learning. represents the first Q value network in transfer learning, represents the second Q value network in transfer learning, represents the first target Q value network in transfer learning, represents the second target Q value network in transfer learning, represents the first Q value network in the source task, represents the second Q value network in the source task, represents the first target Q value network in the source task, Represents the second target Q value network in the source task.

[0169] In one feasible implementation, the SAC algorithm is continuously used to update the parameters of the target task in transfer learning until the performance converges. It should be noted that the number of elements in the action space of the source task and the target task is different. Several zero elements are added to make Allows the parameters in the source task to be directly passed to the parameters in the target task. The algorithm flow is as follows Figure 3 shown.

[0170] In a feasible implementation, a disaster scenario is set to be modeled, and a simulation scenario is modeled using synthetic data. Then, whether the algorithm proposed in the present invention is effective is discussed.

[0171] First, consider a small-scale disaster scenario. In the rectangular area R∈[-1000m,1000m]×[-1200m,1200m], there are two drones, 17 ground base stations, 20 communication users (15 communication users are served by ground base stations, represented by purple dots, and 5 communication users are not served by ground base stations, represented by blue dots), and 5 sensing targets. The base station antenna is tilted at an angle of 8°, the antenna height is 21 meters to 50 meters, and the transmission power is 24dBm. Figure 2 As an example of a post-disaster scenario, there are 7 antennas left to provide communications, and they operate at a low power of 6dBm, as shown in Figure 3Here, 5 communication users are served by the ground base station (purple dots), while the other 15 communication users are not served (blue dots). Two drones serve as emergency ISACs that support communication users in coverage blind spots and sense targets. The bandwidth B is set to 1MHz, the communication-related transmit antenna gain g1 is 5dB, the communication-related receive antenna gain g2 is 15dB, and the signal wavelength λ CU is 0.1m, the transmit antenna gain g3 related to perception is 15dB, the receive antenna gain g4 related to perception is 15dB, and the radar cross section σ r 1m 2 and the echo signal wavelength λ S 0.35m, minimum and maximum transmission power and Set to 0.01W, 0.1W, respectively, the maximum moving distance of the x-coordinate, the maximum moving distance of the y-coordinate, and the maximum moving distance of the z-coordinate max ,y max 、z max 50m, 50m, 5m respectively, the maximum energy value of the drone is EU max 3KJ, the upper limit of penalty index is F max is 100, the communication user CU in the nth time slot m Data rate threshold 6×10 5 ,bit / s, sensing target S in time slot n l Threshold of radar estimated information rate For 10 5 bit / s, the discount factor γ is 0.98, the learning rate η is 0.01, and the target policy entropy is 20, the proportional constant τ is 1 / 50, the bias constant a is 1.8, the penalty term b is 20, and the noise power σ 2 For 10 -11 .

[0172] In order to evaluate the convergence and effectiveness of the proposed SAC algorithm based on transfer learning, the present invention Figure 4 The iteration of the optimization objective of the SAC algorithm based on transfer learning is shown in Figure 2. Figure 4 It can be seen that with the increase of the number of iterations, the objective function value gradually decreases and finally converges, indicating that the algorithm in this paper is effective.

[0173] from Figure 5It can be clearly seen that as the number of iterations increases, the cumulative reward item shows a gradual upward trend. This shows that during the training process, the agent continuously learns and optimizes its decision-making strategy, thereby achieving better performance in each iteration. The increase in cumulative rewards reflects that the agent gradually learns better decisions in a complex environment, thereby effectively improving the success rate of communication and perception tasks and optimizing resource utilization efficiency. This growth trend of cumulative rewards verifies the effectiveness of the adopted algorithm and demonstrates the continuous improvement and optimization capabilities of the algorithm when performing task allocation and resource scheduling in the drone-assisted ISAC system.

[0174] Figure 6 The iterative state of the optimization objective under the SAC algorithm and the SAC algorithm based on transfer learning is shown in Figure 2. Figure 6 It can be seen that as the number of iterations increases, the optimization target gradually decreases. This trend shows that with the continuous iteration of the algorithm, both algorithms can effectively optimize the task allocation and resource scheduling strategies, thereby reducing the overall optimization target value. In contrast, the SAC algorithm based on transfer learning converges faster and converges to a better solution. This means that the algorithm has higher adaptability and efficiency when dealing with dynamic environmental changes. In general, Figure 6 The results in verify the effectiveness of these two algorithms in optimizing task scheduling and resource allocation in UAV-assisted ISAC systems.

[0175] In order to further prove the performance of the algorithm, the present invention Figure 7 The trajectories of the drones after 10,000 iterations of the SAC algorithm before repairing the ground base station are visualized in Figure 1. The trajectories of the two drones are represented in red and green, respectively, and the small solid circles of the corresponding colors represent the paths taken by the drones. The triangle represents the starting point of the drone, while the pentagon represents the end point. Figure 7 It can be seen that the drone flies over almost every communication user and perception target, providing ISAC dual functions. In addition, the drone rarely flies over areas lacking communication users and perception targets. In addition, the drone hovers back and forth over the communication users and perception targets to ensure fairness and QoS of communication and perception.

[0176] The present invention Figure 8 The figure shows the UAV trajectory generated by the SAC algorithm based on transfer learning after 10,000 iterations. Figure 8 As can be seen in Figure 1, the drone successfully adjusted its strategy to provide ISAC services to the updated unserved communication users and perceived targets, while avoiding providing services to the communication users covered by the repaired ground base stations. During the flight, the drone also provided ISAC services by hovering back and forth.

[0177] In summary, the SAC algorithm, especially the transfer learning-based SAC algorithm, shows excellent performance in UAV-assisted ISAC networks, especially for efficient joint task scheduling and resource allocation considering the uncertainty of base station availability.

[0178] The present invention proposes a novel post-disaster UAV-assisted ISAC system, in which the availability of ground base stations changes dynamically. The present invention sets up the deployment of UAVs to supplement failed base stations and monitor potential sensing targets to grasp the situation in the disaster area. The contributions of the present invention are mainly reflected in the following aspects. First, in order to balance the relationship between communication and perception, the present invention proposes two novel indicators, namely, the communication penalty index and the perception penalty index, to ensure wide-area communication, effective perception and fair quality of service (QoS). Under the energy constraint of the UAV, the present invention establishes a joint task scheduling and resource allocation model to minimize the weighted sum of these indicators. Secondly, in order to reduce the time complexity and improve the scalability, the present invention reformulates the optimization problem as a Markov decision process (MDP). When the availability of the base station changes, the corresponding Markov decision process will also be adjusted accordingly. Finally, the present invention proposes an advanced deep reinforcement learning algorithm, the Soft Actor-Critic (SAC), to solve the Markov decision process problem. In order to enhance robustness and shorten training time, the present invention applies transfer learning to the SAC algorithm, transferring the parameters of the trained model to the updated Markov decision process when the base station availability changes. Experimental results with base station locations and configurations show that SAC outperforms the baseline algorithm in communication and perception performance as well as convergence speed. In addition, SAC based on transfer learning exhibits faster convergence speed and better robustness when base station availability changes suddenly.

[0179] Fig. 9 1 is a block diagram of a task scheduling and resource allocation device 300 for a post-disaster UAV interoception system according to an exemplary embodiment. The device 300 is used for a task scheduling and resource allocation method for a post-disaster UAV interoception system. Fig. 9 The device includes a drone deployment module 310, a performance indicator confirmation module 320, an optimization problem module 330, an optimization problem conversion module 340, and a communication and perception module 350. Among them:

[0180] The drone deployment module 310 is used to deploy drones to supplement the failed base stations and monitor potential sensing targets through the deployed drones;

[0181] The performance index confirmation module 320 is used to construct the communication performance index of the UAV, follow the penalty function method in the constraint optimization, and use the communication penalty index as the optimization target under the energy constraint of the UAV;

[0182] Construct the perception performance index of the UAV, follow the penalty function method in constrained optimization, and use the perception penalty index as the optimization target under the energy constraint of the UAV;

[0183] An optimization problem module 330 is used to establish a joint task scheduling and resource allocation model, and to obtain an optimization problem by minimizing a weighted sum of a communication performance index and a perception performance index through the joint task scheduling and resource allocation model;

[0184] An optimization problem conversion module 340 for reformulating the optimization problem as a Markov decision process;

[0185] The communication and perception module 350 is used to obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm SAC model to update the Markov decision process, apply transfer learning to the soft actor-critic algorithm SAC model, and when the base station availability changes, transfer the parameters of the trained model to the updated Markov decision process to obtain the optimal value of the optimization variable, and then perform task scheduling and resource allocation for the post-disaster UAV synaesthesia system.

[0186] Optionally, the drone deployment module 310 is used to obtain I base stations within a predefined service area U; obtain a blank portion of network coverage within the area U;

[0187] Deploy K drones in the blank part of the network coverage in area U, and K drones simultaneously detect L potential sensing targets on the ground;

[0188] Among them, both drones and base stations are equipped with omnidirectional antennas to broadcast information to communication users;

[0189] In each time slot, all drones first fly to the predetermined location and then hover to provide ISAC services.

[0190] Optionally, the performance indicator confirmation module 320 is used to construct the communication performance indicators of the drone; the communication performance indicators of the drone include: communication penalty index signal to interference noise ratio and communication user CU m Data rate

[0191] When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0192] If the data rate reaches the required threshold It is considered that the communication user has received the communication successfully, and vice versa; is the sum of the communication penalty indexes of all communication users;

[0193] in, Indicates the communication user CU in time slot n m The communication penalty index, The communication user CU in time slot n-1 is represented as m The communication penalty index, F max represents the maximum value of the communication penalty index, Represented as the communication user CU in the nth time slot m Minimum value of data rate; overall communication penalty index within time slot n is the sum of the penalty indices of all communicating users.

[0194] Optionally, the performance indicator confirmation module 320 is used to obtain the perception target;

[0195] Construct the perception performance index of the UAV; the perception performance index includes: perception penalty index;

[0196] When the radar estimates the information rate Greater than the set threshold , indicating that the perception is successful, that is, the perception state The value of is 1;

[0197] When a perception target is successfully perceived within a given time slot, its perception penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ;

[0198] in, Indicates the target S is sensed in time slot n l The radar estimated information rate; Indicates the target S is sensed in time slot n l The threshold of radar estimated information rate, Indicates the target S is sensed in time slot n l The state of perception, Indicates the target S is sensed in time slot n l The perceived penalty index, Indicates that the target S is sensed in time slot n-1 l The perceived penalty index.

[0199] Optionally, an optimization problem module 330 is used to establish a joint task scheduling and resource allocation model;

[0200] Get the optimization target, which is to minimize the communication penalty index and perceived penalty index The weighted sum of

[0201] The optimization objective is as shown in the following formula (1):

[0202]

[0203] Among them, the optimization objective includes three decision variables: the transmission power set P UAV , UAV trajectory set u and associated indicator set Ω;

[0204] The constraints that need to be satisfied in the optimization problem include: is the radar pulse power constraint of the UAV, For the drone boundary constraints, as well as is the moving distance constraint of the UAV in a time slot n, EU k,n ≤EU max is the energy constraint of the UAV, α k,j,n ∈{0,1} and is the association indicator constraint, all k, j, n must satisfy the above constraints;

[0205] in, and Indicates the minimum and maximum transmit power, Indicates the number of UAVs in time slot n k The horizontal axis of Indicates that UAV in time slot n-1 k The horizontal axis of Indicates the number of UAVs in time slot n k The vertical coordinate of Indicates that UAV in time slot n-1 k The vertical coordinate of Indicates the number of UAVs in time slot n k Height, Indicates that UAV in time slot n-1 k The height, x max ,y max 、z max are the maximum moving distance of the x-coordinate, y-coordinate, and z-coordinate of the drone in a time slot n, respectively. max Represents the maximum energy value of the drone, α k,j,n Indicates whether it is a UAV k Serving communication users CUj The associated indicator.

[0206] Optionally, an optimization problem conversion module 340 is used to convert the optimization problem into a Markov decision process, represented as a four-element tuple (S, A, P, R);

[0207] Among them, the state space set S in time slot n n Includes location information of all base stations Location information of all drones Location information of all communicating users Location information of all perceived targets And the energy consumption of all drones

[0208] The action set A in time slot n n There are a total of (4+J)×K elements, including the flight distances of all drones Transmit power and associated indicator sets

[0209] Strategy P = P(S n+1 |S n ,A) means in state S n After taking action A, it moves to the new state S n+1 The probability of

[0210] In time slot n, the reward R n Defined as -τ(Y C +Y S )+a, where τ is the proportional constant and a is the bias constant. When the constraints in the optimization problem cannot be met, R n For R n -b, where b is a penalty term. The present invention sets b to a sufficiently large value.

[0211] Optionally, the communication and perception module 350 is used to obtain a reformulated optimization problem

[0212] Combining the principle of maximum entropy reinforcement learning with the advantages of the actor-critic framework, the soft actor-critic algorithm SAC is obtained, and the Markov decision process is updated through the soft actor-critic algorithm SAC;

[0213] Transfer learning is used on the SAC algorithm to obtain SAC based on transfer learning. After multiple iterations of training, the trained parameters of the source task are In transfer learning, the parameters of the target task are initialized using the trained parameters of the source task;

[0214] After initialization, continue to use the SAC algorithm to update the parameters of the target task in transfer learning until the performance converges;

[0215] By using the SAC algorithm based on transfer learning to adapt to the changes in base stations after the disaster, the task scheduling and resource allocation of the post-disaster UAV interawareness system are completed.

[0216] In the embodiments of the present invention, a novel post-disaster drone-assisted ISAC system is considered, in which drones are used to supplement the network capacity of ground base stations that have lost service, while using their radio signals for perception. The present invention applies transfer learning and deep reinforcement learning to learn task scheduling and resource allocation strategies that can quickly adapt to the uncertainty in the availability of ground base stations. Experimental results show that the algorithm outperforms existing algorithms in terms of communication and perception performance and convergence speed. In addition, when the availability of ground base stations changes suddenly, DRL based on transfer learning has faster convergence and better robustness.

[0217] Fig.10 is a structural diagram of a task scheduling and resource allocation device for a post-disaster UAV interawareness system provided by an embodiment of the present invention, such as Fig.10 As shown, the task scheduling and resource allocation equipment of the post-disaster UAV interawareness system may include the above Fig. 9 The task scheduling and resource allocation device of the post-disaster UAV synaesthesia system shown in the figure. Optionally, the task scheduling and resource allocation device 410 of the post-disaster UAV synaesthesia system may include a first processor 2001.

[0218] Optionally, the task scheduling and resource allocation device 410 of the post-disaster drone interawareness system may also include a memory 2002 and a transceiver 2003 .

[0219] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for task scheduling and resource allocation of a post-disaster UAV synaesthesia system, characterized in that: The method comprises: S1. Deploy drones to supplement failed base stations and monitor potential sensing targets through the deployed drones; S2. Construct the communication performance index of the UAV, follow the penalty function method in constrained optimization, and use the communication penalty index as the optimization target under the energy constraint of the UAV; Construct the perception performance index of the UAV, follow the penalty function method in constrained optimization, and use the perception penalty index as the optimization target under the energy constraint of the UAV; S3, establishing a joint task scheduling and resource allocation model, and minimizing the weighted sum of the communication performance index and the perception performance index through the joint task scheduling and resource allocation model to obtain an optimization problem; S4. Reformulate the optimization problem as a Markov decision process; S5. Obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm (SAC) model to update the Markov decision process, and apply transfer learning to the soft actor-critic algorithm (SAC) model; when the base station availability changes, transfer the parameters of the trained soft actor-critic algorithm (SAC) model to the updated Markov decision process, obtain the optimal values ​​of the optimization variables, and then perform task scheduling and resource allocation for the post-disaster UAV synaesthesia system.

2. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 1 is characterized in that: In S1, a drone is deployed to supplement the failed base station, and at the same time, the deployed drone is used to monitor potential sensing targets, including: Obtain I base stations within a predefined service area U; obtain the blank portion of network coverage within area U; Deploy K drones in the blank part of the network coverage in area U, and K drones simultaneously detect L potential sensing targets on the ground; Among them, both drones and base stations are equipped with omnidirectional antennas to broadcast information to communication users; In each time slot, all drones first fly to the predetermined location and then hover to provide ISAC services.

3. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 2 is characterized in that: In S2, the communication performance index of the UAV is obtained, and the penalty function method in the constraint optimization is followed to take the communication penalty index as the optimization target under the energy constraint of the UAV, including: Construct the communication performance index of the UAV; the communication performance index of the UAV includes: communication penalty index, signal to interference noise ratio and communication user CU m Data rate When a communication user successfully receives a communication in a given time slot, its communication penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ; If the data rate reaches the required threshold It is considered that the communication user has received the communication successfully, and vice versa; is the sum of the communication penalty indexes of all communication users; in, Indicates the communication user CU in time slot n m The communication penalty index, The communication user CU in time slot n-1 is represented as m The communication penalty index, F max represents the maximum value of the communication penalty index, Represented as the communication user CU in the nth time slot m Minimum value of data rate; overall communication penalty index within time slot n is the sum of the penalty indices of all communicating users.

4. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 3 is characterized in that: In S2, the perception performance index of the UAV is obtained, and the penalty function method in the constraint optimization is followed to use the perception penalty index as the optimization target under the energy constraint of the UAV, including: Acquire perceived targets; Constructing a perception performance index of a drone; the perception performance index includes: a perception penalty index; When the radar estimates the information rate Greater than the set threshold When , it indicates that the perception is successful, that is, the perception state The value of is 1; When a perception target is successfully perceived within a given time slot, its perception penalty index is reset to 0; otherwise, the communication penalty index In its historical value Add 1 to the base until the upper limit F is reached max ; in, Indicates the target S is sensed in time slot n l The radar estimated information rate; Indicates the target S is sensed in time slot n l The threshold of radar estimated information rate, Indicates the target S is sensed in time slot n l The state of perception, Indicates the target S is sensed in time slot n l The perceived penalty index, Indicates that the target S is sensed in time slot n-1 l The perceived penalty index.

5. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 4 is characterized in that: In S3, a joint task scheduling and resource allocation model is established, and a weighted sum of a communication performance index and a perception performance index is minimized by the joint task scheduling and resource allocation model, including: Establish joint task scheduling and resource allocation models; Get the optimization goal, which is to minimize the communication penalty index and perceived penalty index The weighted sum of Then the optimization objective is as shown in the following formula (1): The optimization objective includes three decision variables: the transmission power set P UAV , UAV trajectory set u and associated indicator set Ω; The constraints that need to be satisfied in the optimization problem include: is the radar pulse power constraint of the UAV, For the drone boundary constraints, as well as is the moving distance constraint of the UAV in a time slot n, EU k,n ≤EU max is the energy constraint of the UAV, α k,j,n ∈{0,1} and is the association indicator constraint, all k, j, n must satisfy the above constraints; in, and Indicates the minimum and maximum transmit power, Indicates the number of UAVs in time slot n k The horizontal axis of Indicates that UAV in time slot n-1 k The horizontal axis of Indicates the number of UAVs in time slot n k The vertical coordinate of Indicates that UAV in time slot n-1 k The vertical coordinate of Indicates the number of UAVs in time slot n k Height, Indicates that UAV in time slot n-1 k The height, x max ,y max 、z max are the maximum moving distance of the x-coordinate, y-coordinate, and z-coordinate of the drone in a time slot n, respectively. max Represents the maximum energy value of the drone, α k,j,n Indicates whether it is a UAV k Serving communication users CU j The associated indicator.

6. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 5 is characterized in that: In S4, the optimization problem is reformulated as a Markov decision process, including: Convert the optimization problem into a Markov decision process, represented as a four-element tuple (S, A, P, R); Among them, the state space set S in time slot n n Includes location information of all base stations Location information of all drones Location information of all communicating users Location information of all perceived targets And the energy consumption of all drones The action set A in time slot n n There are a total of (4+J)×K elements, including the flight distances of all drones Transmit power and associated indicator sets Strategy P = P(S n+1 |S n ,A) means in state S n After taking action A, it moves to the new state S n+1 probability; In time slot n, the reward R n Defined as -τ(Y C +Y S )+a, where τ is the proportional constant and a is the bias constant. When the constraints in the optimization problem cannot be met, R n For R n -b, where b is a penalty term. The present invention sets b to a sufficiently large value.

7. The task scheduling and resource allocation method of the post-disaster UAV synaesthesia system according to claim 6 is characterized in that: In S5, a reformulated optimization problem is obtained; a soft actor-critic algorithm SAC model is constructed and trained to update the Markov decision process, and transfer learning is applied to the soft actor-critic algorithm SAC model. When the availability of the base station changes, the parameters of the trained model are transferred to the updated Markov decision process to obtain the optimal value of the optimization variable. Then, the task scheduling and resource allocation of the post-disaster UAV synaesthesia system are performed, including: Get the reformulated optimization problem Combining the principle of maximum entropy reinforcement learning with the advantages of the actor-critic framework, a soft actor-critic algorithm SAC model is obtained, and the Markov decision process is updated through the soft actor-critic algorithm SAC model; Transfer learning is used on the SAC model to obtain a SAC model based on transfer learning. After multiple iterations of training, the trained parameters of the source task are In transfer learning, the parameters of the target task are initialized using the trained parameters of the source task; After initialization, continue to use the SAC model to update the parameters of the target task in transfer learning until the performance converges; obtain the optimal value of the optimization variable; wherein the optimization variable is the decision variable in the optimization target; By using the SAC model based on transfer learning to adapt to the changes of base stations after the disaster, the task scheduling and resource allocation of the post-disaster UAV interawareness system are completed.

8. A task scheduling and resource allocation device for a post-disaster UAV synaesthesia system, the task scheduling and resource allocation device for a post-disaster UAV synaesthesia system is used to implement the task scheduling and resource allocation method for a post-disaster UAV synaesthesia system as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A drone deployment module is used to deploy drones to supplement failed base stations and monitor potential sensing targets through the deployed drones; The performance index confirmation module is used to construct the communication performance index of the UAV, following the penalty function method in constrained optimization, and taking the communication penalty index as the optimization target under the energy constraint of the UAV; Construct the perception performance index of the UAV, follow the penalty function method in constrained optimization, and use the perception penalty index as the optimization target under the energy constraint of the UAV; An optimization problem module, used for establishing a joint task scheduling and resource allocation model, and obtaining an optimization problem by minimizing a weighted sum of a communication performance index and a perception performance index through the joint task scheduling and resource allocation model; Optimization problem transformation module, used to reformulate the optimization problem into a Markov decision process; The communication and perception modules are used to obtain the reformulated optimization problem; construct and train the soft actor-critic algorithm (SAC) model to update the Markov decision process, and apply transfer learning to the soft actor-critic algorithm (SAC) model; when the base station availability changes, the parameters of the trained soft actor-critic algorithm (SAC) model are transferred to the updated Markov decision process, and after obtaining the optimal values ​​of the optimization variables, the task scheduling and resource allocation of the post-disaster UAV synaesthesia system are carried out.

9. A task scheduling and resource allocation device for a post-disaster UAV interawareness system, characterized in that: The task scheduling and resource allocation equipment of the post-disaster UAV interawareness system includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images