Multi-UAV-Assisted Wireless Caching Network Delay Optimization Method and System

Abstract

The present invention discloses a method and system for optimizing the delay of a multi-UAV assisted wireless caching network, which relates to the technical field of wireless caching network delay optimization. The present invention includes: constructing a wireless caching network architecture model in which multiple UAVs and a ground base station cooperate based on the remaining battery ratio of the UAVs; introducing the age of information as an index to measure the freshness of file content based on the wireless caching network architecture model, and improving the caching update mechanism. By constructing a wireless caching architecture in which multiple UAVs cooperate with the ground base station based on the remaining battery ratio of the UAVs, the present invention improves the file distribution efficiency; multi-UAV cooperative caching can more effectively share the caching burden than single-UAV caching, avoiding service interruption or delay caused by power or caching capacity limitations; the designed weighted K-means algorithm considering user location, activity and importance ensures that the UAVs are deployed near active and important users.

Description

Technical Field

[0001] The present invention relates to the technical field of wireless caching network delay optimization, and specifically to a method and system for optimizing the delay of a multi-UAV assisted wireless caching network. Background Art

[0002] With the continuous growth of network application requirements, traditional wireless communication networks are facing increasingly severe challenges. With the increase in the number of users and the explosion of data traffic, the traditional base station architecture cannot effectively cope with the huge data transmission requirements, resulting in problems such as too long transmission paths and excessive response delays. Especially in environments with wide coverage or uneven user distribution, the burden on the base stations increases, leading to a significant decline in network performance.

[0003] To solve this problem, in recent years, UAV-assisted wireless caching networks, as a new type of network architecture, have gradually attracted attention. Due to the advantages of flexible deployment and fast response of UAVs, they can provide file distribution services for users in the air, significantly reducing the coverage blind spots and transmission delays faced by traditional base stations. Although this architecture has shown advantages in improving transmission efficiency and network response speed, the caching capacity of a single UAV is limited and restricted by battery power. Moreover, most of the traditional multi-UAV location deployment work only considers the distance of users, ignoring the impact of user activity and importance on file distribution services. Therefore, the present invention proposes a method and system for optimizing the delay of a multi-UAV assisted wireless caching network. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for optimizing the delay of a multi-UAV assisted wireless caching network. By means of the collaborative working mechanism between multiple UAVs and the base station, the file distribution efficiency and user response speed are improved. Compared with a single UAV, the collaborative work of multiple UAVs can effectively share the caching burden, avoiding service interruptions or delays caused by battery power or caching capacity limitations, and improving the resource utilization efficiency through a collaborative caching mechanism; by considering a caching sharing mode based on the remaining battery power ratio of UAVs, the remaining battery power limitation of UAVs in practical applications is fully considered, improving the overall caching efficiency of the system; at the same time, based on three factors: user location, user activity, and user importance, a weighted K-means algorithm is proposed to determine the deployment positions of multiple UAVs, improving the overall user experience; finally, by dynamically adjusting the file caching placement strategy, the system delay can be significantly reduced under multiple constraints, while ensuring the timeliness of file content. This method can adapt to complex multi-UAV assisted wireless caching network application scenarios, providing an optimized technical solution for environments such as smart cities and remote monitoring.

[0005] According to the first aspect of the present invention, to achieve the above object, the present invention provides the following technical solution: a method for optimizing the delay of a multi-UAV assisted wireless caching network. The multi-UAV assisted wireless caching network includes users, a base station within a cellular network, and UAVs that respond to user requests. The method includes the following steps:

[0006] Users in the cellular network area are divided into multiple clusters, and each cluster corresponds to a UAV for service. Each UAV is equipped with a caching unit, and the caching unit on the UAV is used to cache the file content published by the ground source node. The UAV corresponding to the user cluster transmits the cached file content to the user through a channel according to the user request;

[0007] If the file content is in the caching unit of the UAV in the adjacent cluster of the user, the caching unit of the UAV in the adjacent cluster is transmitted to the caching unit of the UAV in the user corresponding cluster, and the UAV in the user corresponding cluster transmits it to the user through the channel;

[0008] Taking the caching capacity of the UAV, the caching location of the file content, the freshness requirement of the file content, and the stability of the multi-UAV assisted wireless caching network as constraints, a deep reinforcement learning algorithm is used for optimization. Taking the maximum response rate of the multi-UAV assisted wireless caching network to the user according to the user request for the file content as the goal, the network delay of the multi-UAV assisted wireless caching is optimized.

[0009] Further, the users in the cellular network area are divided into two clusters, namely cluster one and cluster two, and each cluster is equipped with a UAV and UAV for service. The base station serves as a backup transmission node and can provide file transmission services for remote users when the UAV cache is insufficient;

[0010] In addition, there is a source node in the cellular network, which is used to monitor and generate the latest file version in real time; in each time slot , UAV , UAV , and the base station respectively wirelessly transmit the dynamic file content generated by the cache source node and deliver the file to the user as needed.

[0011] Further, UAV and UAV will allocate the number of cached files for the files cached at the UAV end according to their own power in the current time slot, and the allocation ratio is .

[0012] Further, according to the user request for the file content, the UAV corresponding to the user cluster transmits it to the user through the channel, which specifically includes the following situations:

[0013] (41) If the file content requested by the user in Cluster 1 is cached at the UAV side, it is provided to the user by the UAV through a single channel;

[0014] (42) If the file content requested by the user in Cluster 1 is cached at the UAV side, it is transmitted from the UAV to the UAV , and then distributed to the user by the UAV ;

[0015] (43) If the file content requested by the user in Cluster 1 is cached at the base station side, it is provided to the user by the base station through a single channel.

[0016] Furthermore, the channel between the user and the UAV is modeled as a line-of-sight or non-line-of-sight connected air-to-ground link. The average service rate of the UAV to the users within its service range is expressed as:

[0017]

[0018] In the formula, is the service rate of the UAV to the user , is the user activity, indicating the probability that the user sends a request; is the user importance, indicating the priority of the user's request; is the available bandwidth of the UAV , is the transmit power of the UAV, is the UAV and the user channel gain between, is the unified size of a single file, is Gaussian noise, is the UAV number of users in the service range. In the scenario of two adjacent UAVs, ;

[0019] The transmission link between the source node and the UAV also considers a line-of-sight or non-line-of-sight connected air-to-ground link model. The file refresh rate from the source node to the UAV is expressed as:

[0020]

[0021] In the formula, , are the source node to the UAV The transmission bandwidth and transmission power, is the channel gain between the source node and the UAV and is the unified size of a single file, and

[0022] is the Gaussian noise; The transmission link between UAV and UAV

[0023]

[0024] In the formula, is the transmission bandwidth between the two UAVs, is the transmission power between the two UAVs, is the additional attenuation factor of the line-of-sight link, and are the positions of UAV and UAV respectively, is the unified size of a single file, and

[0025] The channel between the user and the base station is a ground-to-ground connection, modeled based on path loss and Rayleigh fading. The average service rate from the base station to the user in the cluster

[0026]

[0027] In the formula, is the service rate from the base station to the user is the user activity, indicating the probability that the user sends a request; is the user importance, indicating the priority of the user's request; is the available bandwidth of the base station, is the transmission power of the base station, is the channel gain between the base station and the user is the unified size of a single file, is the Gaussian noise, is the number of users within the service range of UAV

[0028]

[0029] The transmission link between the source node and the base station is also modeled based on path loss and Rayleigh fading. The file refresh rate from the source node to the base station is:

[0029]

[0030] In the formula, and are the transmission bandwidth and transmission power from the source node to the base station respectively, is the channel gain between the source node and the base station, is the unified size of a single file, is the Gaussian noise.

[0031] Furthermore, before the UAV or the base station sends a file, it will check the freshness of the file and introduce the age of information as an index to measure the freshness of the file content. Taking the user requests in Cluster 1 as an example, the specific situation is as follows:

[0032] (61) Introduce an M / M / 1 queue to represent the UAV service process and capture the additional delay load brought by cache refreshing. The average refresh probability of the file at the UAV end within the time slot

[0033]

[0034] In the formula, is the probability that a request for a file initiated by a user in Cluster 1 at the UAV end triggers cache refreshing within the time slot , is the request rate of users in Cluster 1 for a single file at the UAV end, is the total request rate of all users in Cluster 1, is the popularity of the file within the time slot , represents the file cache set of the UAV within the time slot , is the cache refresh window size, is the base of the natural logarithm, is the file refresh rate from the source node to the UAV end;

[0035] The average age of information of users in Cluster 1 receiving the file at the UAV end within the time slot

[0036]

[0037] In the formula, is the average service rate from the UAV end to all users in Cluster 1, is the file refresh rate from the source node to the UAV is the cache refresh window size, is the average service delay from the UAV to the users in Cluster 1 for a single file at the UAV request rate, is the base of the natural logarithm;

[0038] The UAV average service delay to the users in Cluster 1 is expressed as:

[0039]

[0040] In the formula, is the average refresh probability of the file at the UAV requested by the users in Cluster 1 within the time slot is the average service rate from the UAV to all users in Cluster 1, is the file refresh rate from the source node to the UAV is the total request rate of the users in Cluster 1 for the cached files at the UAV is the file refresh rate from the source node to the UAV is the total request rate of all users in Cluster 1, is the popularity of the file is the total request rate of all users in Cluster 1, is the file within the time slot popularity, represents the file cache set of the UAV within the time slot ;

[0041] The UAV average probability that a file request initiated by a user in Cluster 1 triggers cache refresh is expressed as:

[0042]

[0043] In the formula, is the probability that a request for a file initiated by a user in Cluster 1 at the UAV triggers cache refresh within the time slot is the request rate of a single file at the UAV by the users in Cluster 1, is the total request rate of all users in Cluster 1, is the request rate of a single file at the UAV is the total request rate of all users in Cluster 1, is the file within the time slot popularity, represents the UAV within the time slot The file cache set within is the cache refresh window size, is the base of the natural logarithm, is the file refresh rate from the source node to the UAV ;

[0044] The average age of information received by users in Cluster 1 from the UAV end file is:

[0045]

[0046] In the formula, is the average service rate of the UAV to all users in Cluster 1, is the transmission rate between the UAV and the UAV ; is the file refresh rate from the source node to the UAV ; is the cache refresh window size, is the request rate of users in Cluster 1 for a single file at the UAV end, ; is the base of the natural logarithm;

[0047] The average service delay from the UAV end to users in Cluster 1 is expressed as:

[0048]

[0049] In the formula, is the average refresh probability of the file at the UAV end requested by users in Cluster 1 within the time slot ; is the average service rate of the UAV to all users in Cluster 1, is the transmission rate between the UAV and the UAV ; is the file refresh rate from the source node to the UAV ; is the total request rate of users in Cluster 1 for the cached files on the UAV side, is the total request rate of all users in Cluster 1, is the file at the time slot popularity, represents the UAV at the time slot The file cache set within

[0050] (62)Introduce the M / M / 1 queue to represent the base station service process and capture the additional delay load brought by cache refreshing. At the base station end, the average probability that a file request initiated by a user in cluster one triggers cache refreshing is expressed as:

[0051]

[0052] In the formula, is at the base station end, within time slot for a file request initiated by a user in cluster one that triggers cache refreshing. is the request rate of a single file at the base station end by users in cluster one, is the total request rate of all users in cluster one, is the popularity of file within time slot , represents the file cache set within the base station within time slot is the cache refreshing window size, is the base of the natural logarithm, is the file refreshing rate from the source node to the base station end;

[0053] The average age of information of users in cluster one receiving the file at the base station end within time slot is:

[0054]

[0055] In the formula, is the average service rate from the base station to all users in cluster one, is the file refreshing rate from the source node to the base station end, is the cache refreshing window size, is the request rate of a single file at the base station end by users in cluster one, is the base of the natural logarithm;

[0056] The average service delay from the base station to users in cluster one is expressed as:

[0057]

[0058] In the formula, is the base station requested by the users in Cluster 1 The average refresh probability of the file at the end during the time slot is is the base station The average service rate to all users in Cluster 1 is the file refresh rate from the source node to the base station is the total request rate of the users in Cluster 1 for the cached files on the base station side is the total request rate of all users in Cluster 1 is the file During the time slot Popularity Indicates the base station During the time slot The file cache set within

[0059] Furthermore, the users in Cluster 2 are analyzed in the same way as in (61) and (62). The average age of information of the users in Cluster 2 for the files received from the UAV within the time slot , UAV and the base station The average age of information of the end file is respectively:

[0060]

[0061]

[0062]

[0063] In the formula, is the average transmission rate from the UAV to the users in Cluster 2 is the UAV and UAV The transmission rate between is the base station The average transmission rate to the users in Cluster 2 , and are respectively the file refresh rates from the source node to the UAV , UAV and the base station is the cache refresh window , and are respectively the requests of the users in Cluster 2 for the UAVs​​ Terminal, UAV Terminal and base station Single file at the terminal The request rate of is the total request rate of users in cluster two, is the file at time slot popularity, , and are the UAV , UAV and base station file cache sets within time slot ;

[0064] The average service delay of UAV , UAV and base station to users in cluster two is expressed as:

[0065]

[0066]

[0067]

[0068] Wherein, , and are the average probabilities of cache refresh triggered by file requests initiated by users in cluster two at the UAV , UAV and base station terminals; , and are the probabilities of cache refresh triggered by a request for a file initiated by users in cluster two at the UAV , UAV and base station terminals within time slot ; , and are the request rates of single files at the UAV terminal, UAV terminal and base station terminal of users in cluster two, is the total request rate of users in cluster two, is the file at time slot popularity, , and are the file cache sets of the UAV , the UAV and the base station within the time slot ; , and are respectively the total request rates of the users in cluster two for the cached files at the UAV , the UAV and the base station ; is the average transmission rate from the UAV to the users in cluster two, is the transmission rate between the UAV and the UAV ; is the average transmission rate from the base station to the users in cluster two, , and are respectively the file refresh rates from the source node to the UAV , the UAV and the base station ;

[0069] The average service delay of the overall wireless caching network within the time slot is expressed as:

[0070] .

[0071] Furthermore, with the cache capacity of the UAV, the cache location of the file content, the freshness requirement of the file content, and the stability of the multi-UAV assisted wireless caching network as constraints, a deep reinforcement learning algorithm is used for optimization, as follows:

[0072] (81) Establish decision variables: Define the positions of the UAV and the UAV as and respectively, and a set of decision variables represents the cache location of the file in the time slot :

[0073]

[0074] wherein, represents the UAV , represents the UAV , represents the base station ; is defined as the time slot The caching location of each file, is the total number of files;

[0075] (82) Establish an optimization objective. The goal of the optimization process is to determine the UAV deployment location and find a set of file caching decisions under various constraints to minimize the system delay. The specific objective function is defined as the long-term system delay:

[0076]

[0077] where, is the average service delay in time slot , is the expectation operator, indicating taking the expectation of the random variable in the parentheses, is the time step, represents the limit operation;

[0078] (83) Set the constraints for the delay optimization decision, specifically:

[0079] (831) UAV cache capacity constraint: The cache capacity of the UAV is limited, and the files cached on the UAV cannot exceed its maximum cache number , and the formula is expressed as:

[0080]

[0081] where, is the unified size of all files, represents that in time slot file is cached on UAV , is the total number of files;

[0082] (832) Caching decision constraint: Each file can only be cached on one node, and the formula is expressed as:

[0083]

[0084] (833) File content freshness constraint: The users in cluster one and cluster two have timeliness requirements for the obtained files, and the files obtained from the UAV or the base station cannot exceed the maximum limit of the file information age when reaching the user side , and the formula is expressed as:

[0085]

[0086]

[0087] where, 、 They are the clusters The request rates of users in the drone side and the base station side for cached files, and They are the drone and the base station The file cache sets within the time slot respectively, and They are the clusters The request rates of users in the drone side and the base station side for a single file respectively, and They are the average age of information of files received by users in the cluster from the drone side and the base station side within the time slot respectively;

[0088] 834 System stability requirement: In the adopted M / M / 1 service system, the file request arrival rate at the drone and base station sides is less than the file request departure rate to ensure system stability. The formula is expressed as:

[0089]

[0090] In the formula, and They are the request rates of users in the drone side and the base station side for cached files, respectively, and They are the average probabilities of cache refresh triggered by file requests initiated by users in the cluster at the drone side and the base station side respectively, and They are the average transmission rates from the drone and the base station to users in the cluster respectively, is the transmission rate between the drone and the drone and They are the file refresh rates from the source node to the drone and the base station respectively;

[0091] 835 The optimization problem and its constraint conditions are expressed in the following form:

[0092]

[0093] Wherein, and are the positions of the UAV and the UAV respectively, is the time slot and the cache positions of each file, indicates that the file is cached at the node in the time slot , represents the UAV , represents the UAV , represents the base station ; is the long-term time delay of the system; C1 indicates that the cache capacity of the UAV is limited, and the files cached on the UAV cannot exceed its maximum cache number , is the unified size of all files, indicates that in the time slot the file is cached on the UAV , is the total number of files; C2 indicates that each file can only be cached at one node; C3 and C4 indicate that the files obtained from the UAV or the base station cannot exceed the maximum age limit of file information when reaching the user side; and are respectively the request rates of the users in the cluster for the files cached at the UAV side and the base station side, and are respectively the file cache sets of the UAV and the base station in the time slot , and are respectively the request rates of the users in the cluster for a single file cached at the UAV side and the base station side, and are respectively the request rates of the users in the cluster for the files received from the UAV side and the base station side within the time slot End file The average information age; C5 and C6 represent the drones And the base station The arrival rate of file requests at the end must be less than the departure rate of file requests; 、 Are respectively the clusters The request rate of the users in the cluster for the cached files at the end of the drone End, base station End cache file request rate, 、 Are respectively at the drone 、Base station End, cluster The average probability of cache refresh triggered by file requests initiated by users 、 Are respectively the drones 、Base station To the cluster The average transmission rate of the users in Is the drone And the drone The transmission rate between 、 Are respectively the file refresh rates from the source node to the drone 、Base station End

[0094] Furthermore, the optimization problem in step (835) is split into two sub-problems, including: the drone position deployment problem and the file cache decision problem, specifically as follows:

[0095] (91) Use the weighted K-means algorithm based on user location, user activity, and user importance to determine the drone deployment location, specifically:

[0096] (91.1) Initialization: Randomly generate the drone Initial position ;

[0097] (91.2) Calculate the distance from each user To the drone Distance:

[0098]

[0099] In the formula, Is the user Location, Is the total set of users, Is the total number of users;

[0100] (91.3) The user Allocate to the nearest drone , where ;

[0101] (91.4) Recalculate the drone position based on the user's location, activity, and importance:

[0102]

[0103] In the formula, is the user 's location; is the user's activity, indicating the probability that the user sends a request; is the user importance, indicating the priority of the user's request; is the total set of users in the cluster , is the total number of users in the cluster ;

[0104] (91.5) Termination condition: Continuously iterate steps (91.1) to (91.4) until the sum of the position changes of the drone and the drone in the previous and current positions is less than the threshold ;

[0105] (92) Use a deep Q-network to solve the file caching decision problem. The deep Q-network algorithm includes the following algorithm steps:

[0106] (92.1) Initialization: Initialize the state space, action space, and target reward function of the DQN, randomly initialize the parameters of the Q-network , and create a target Q-network;

[0107] (92.2) State definition: The system state consists of the file request probability vector in the current time slot, the remaining battery power of the drone in the current time slot, and the caching policy in the previous time slot;

[0108] (92.3) Action selection: The action represents the caching policy in the current time slot, , which determines the file to be cached to the drone or the base station in the time slot ;

[0109] (92.4) Reward calculation: The reward function is defined according to the optimization objective and constraints, including the system average delay Minimization, and by introducing penalty terms to ensure the satisfaction of constraint conditions, specifically cache capacity limitation, cache location limitation, file freshness requirement, and system stability constraint;

[0110] The reward function expression is:

[0111]

[0112] Wherein, is the average service delay of the system at time slot ; represents the UAV cache capacity constraint, ensuring that the files cached on the UAV do not exceed its maximum cache number ; represents the cache decision constraint, ensuring that each file can only be cached at one node; represents the file content freshness constraint at the UAV side, ensuring that the file obtained from the UAV side does not exceed the maximum bound of the file information age when reaching the user side ; represents the file content freshness constraint at the base station side, ensuring that the file obtained from the base station side does not exceed the maximum bound of the file information age when reaching the user side ; represents the system stability constraint at the UAV side, ensuring that the file request arrival rate at the UAV side is less than the file request departure rate; represents the system stability constraint at the base station side, ensuring that the file request arrival rate at the base station side is less than the file request departure rate, is the penalty coefficient, and the max function represents taking the maximum value in the parentheses, is the unified size of all files, is the UAV cache capacity, represents the cache location of file at time slot ; is the total number of files, 、 are respectively the request arrival rates of users in cluster for the files cached at the UAV side and the base station side, 、 are respectively the UAV 、the base station at time slot The file cache set inside , are respectively the request rate of a single file at the user side of the drone and the base station side; , are respectively the average age of information of the files received by the user from the drone side and the base station side within a time slot for the file ; is the maximum bound of the file age of information, , are respectively the average probability of triggering cache refresh when a file request is initiated by the user at the drone and base station sides for the cluster , are respectively the average transmission rate from the drone and the base station to the users in the cluster; is the transmission rate between the drone and the drone ; , are respectively the file refresh rates from the source node to the drone and the base station ;

[0113] (92.5) Network update: According to the experience replay mechanism, randomly sample a small batch of sample sets for training, and use the target Q - value to update the parameters of the Q - network ;

[0114] (92.6) Action execution: Execute the selected action , update the system state , and record the reward ;

[0115] (92.7) Termination condition: Continuously iterate steps (92.1) to (92.6) until the termination condition for Q - value convergence is met;

[0116] (92.8) After completing the optimization solution, when using the trained DQN to obtain the file cache policy, by inputting the current system state into the trained Q - network, calculate the Q - value of each action, and select the action with the maximum Q - value , the selected action ​This is the optimal caching strategy for the current time slot.

[0117] According to the second aspect of the present invention, the present invention provides a multi-UAV assisted wireless caching network delay optimization system for implementing the above-mentioned multi-UAV assisted wireless caching network delay optimization method, including:

[0118] The first construction module is used to construct a wireless caching network architecture model for the collaborative work of multiple UAVs and a ground base station based on the remaining battery power ratio of the UAVs, so as to improve the file distribution efficiency and user response speed;

[0119] The update mechanism improvement module is used to introduce the age of information as a key indicator of file content freshness based on the wireless caching network architecture model to improve the caching update mechanism;

[0120] The second construction module determines the UAV deployment location based on the weighted K-means algorithm of user location, user activity and user importance to improve the overall user experience;

[0121] The third construction module is used to construct an intelligent optimization model, and adopts a deep reinforcement learning algorithm to dynamically adjust the caching placement strategy to obtain the optimal caching strategy.

[0122] The present invention has at least the following beneficial effects:

[0123] 1. By combining the collaborative work mechanism between multiple UAVs and the base station, the present invention successfully improves the file distribution efficiency and user response speed, greatly optimizes the delay performance of the wireless caching network. Compared with a single UAV, the collaborative work of multiple UAVs can effectively share the caching burden, avoid service interruption or delay caused by power or caching capacity limitations, and improve the resource utilization efficiency through the collaborative caching mechanism.

[0124] 2. The present invention adopts a sharing caching mode based on the remaining battery power ratio of the UAVs in the caching strategy, fully considering the remaining battery power limitation of the UAVs, significantly improving the caching efficiency of the system, and solving problems such as limited capacity of a single UAV and remaining battery power limitation in the traditional network.

[0125] 3. By combining three factors of user location, activity and importance, the present invention optimizes the UAV deployment location by using the weighted K-means algorithm, enabling the system to more accurately meet user needs and improving the overall user experience.

[0126] 4. The present invention introduces the age of information as a key indicator to measure the freshness of file content, improves and optimizes the caching update mechanism, and ensures timely response to user needs and maintains a high-timeliness file distribution service.

[0127] 5. By dynamically adjusting the file cache placement strategy, the present invention can not only significantly reduce the system latency under multiple constraints, but also ensure the timeliness of file content, adapt to complex multi-UAV-assisted wireless cache network application scenarios, and provide an efficient, reliable and timely optimized technical solution for environments such as smart cities and remote monitoring.

[0128] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] Figure 1 is a flowchart of the optimization method described in the present invention;

[0130] Figure 2 is a system scenario diagram of the optimization method described in the present invention;

[0131] Figure 3 is a performance comparison test diagram between the optimization method described in the present invention and traditional algorithms. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0132] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0133] Embodiment 1:

[0134] Please refer to Figure 1 , the present invention provides a technical solution: a method for optimizing the latency of a multi-UAV-assisted wireless cache network, including users, base stations in a cellular network, and UAVs that respond to user requests according to user requests, and includes the following steps:

[0135] S1. Construct a wireless cache network architecture model for the collaborative work of multiple UAVs and ground base stations based on the remaining battery power ratio of the UAVs to improve the file distribution efficiency and user response speed, specifically as follows:

[0136] S1-1 Construct a wireless cache network architecture model assisted by multiple UAVs. In a typical wireless cache cellular network, users are randomly distributed, and these users are far from the base station; multiple UAVs are deployed in the air to serve ground users; for simplicity of analysis, without loss of generality, we consider two adjacent UAVs to carry out system modeling, and a similar analysis method is also applicable to larger-scale scenarios, and thus applicable to the case of multiple UAV deployments. The users in the entire area are divided into at least two clusters, namely Cluster 1 and Cluster 2. Cluster 1 is equipped with UAVs Provide services, and Cluster 2 is equipped with drones Provide services. Each drone is equipped with a caching unit that can cache the file content published by the ground source node; the drone can quickly respond to user requests by caching some files. The base station serves as a backup transmission node and can provide file transmission services to remote users when the drone's cache is insufficient; in addition, there is a source node in the cellular network responsible for real-time monitoring of the surrounding environment and generating the latest file version; in each time slot , two drones and the base station respectively transmit the dynamic file content generated by the source node through wireless transmission according to the set caching strategy, and deliver the files to users on demand;

[0137] S1-2 In the considered drone 、drone caching scenario, drones and drones will allocate the caching quantity of the files cached at the drone end according to their own battery levels in the current time slot 、 , and the allocation ratio is . By cooperative caching, duplicate cached files are avoided, and the overall caching efficiency is improved; when the remaining battery of one of the drones runs out, a fully charged drone is used to replace it;

[0138] S1-3 Please refer to Figure 2 , in the wireless caching network architecture model assisted by multiple drones, if the file content requested by the users in Cluster 1 is cached at the drone end (drone 1 cache), then it is provided to the users by the drone through a single channel; if the file content requested by the users in Cluster 1 is cached at the drone end (drone 2 cache), then it is transmitted from the drone to the drone , and then delivered to the users by the drone ; if the file content requested by the users in Cluster 1 is cached at the base station end (base station cache), then it is provided to the users by the base station through a single channel;

[0139] Before the drone or the base station delivers the file, it checks the freshness of the file. If the age of the file information is lower than the set threshold, the file is directly provided to the users; if the age of the information exceeds the threshold, the drone or the base station will obtain the latest version of the file from the source node, refresh the cache and then deliver it to the users;

[0140] S1-4 In the wireless caching network architecture model assisted by multiple drones, the channel between the users and the drones is modeled as a line-of-sight or non-line-of-sight connected air-to-ground connection. The average service rate of the drone to the users within its service range can be expressed as:

[0141]

[0142] Wherein, is the service rate of the UAV to the user ; is the activity of the user, indicating the probability that the user sends a request; is the user importance, indicating the priority of the user sending a request; is the UAV 's available bandwidth, is the transmission power of the UAV, is the UAV and the user channel gain between, is the unified size of a single file, is Gaussian noise, is the UAV number of users in the service range (in the scenario of two adjacent UAVs considered in this embodiment, );

[0143] The transmission link between the source node and the UAV also considers the line-of-sight or non-line-of-sight connected air-to-ground connection model. The file refresh rate from the source node to the UAV can be expressed as:

[0144]

[0145] Wherein, , are the transmission bandwidth and transmission power from the source node to the UAV respectively, is the channel gain between the source node and the UAV ; is the unified size of a single file, is Gaussian noise;

[0146] The transmission link between the UAV and the UAV considers the line-of-sight connected air-to-air connection model. The transmission rate between the two UAVs can be expressed as:

[0147]

[0148] Wherein, is the transmission bandwidth between the two UAVs, is the transmission power between the two UAVs, is the additional attenuation factor of the line-of-sight link, and are the positions of the unmanned aerial vehicle (UAV) and the UAV respectively, is the unified size of a single file, and is Gaussian noise;

[0149] The channel between the user and the base station is a ground-to-ground connection, modeled based on path loss and Rayleigh fading. The average service rate from the base station to the users in the cluster can be expressed as:

[0150]

[0151] where is the service rate from the base station to the user respectively, is the user activity, representing the probability that the user sends a request; is the user importance, representing the priority of the user's request; is the available bandwidth of the base station, is the transmit power of the base station, is the channel gain between the base station and the user respectively, is the unified size of a single file, and is Gaussian noise, is the UAV and is the number of users within the service range of the UAV;

[0152] The transmission link between the source node and the base station is also modeled based on path loss and Rayleigh fading. The file refresh rate from the source node to the base station is:

[0153]

[0154] where and are the transmission bandwidth and transmission power from the source node to the base station respectively, is the channel gain between the source node and the base station, is the unified size of a single file, and is Gaussian noise;

[0155] S2. Introduce the age of information as a key metric to measure the freshness of file content based on the wireless caching network architecture model, and improve the cache update mechanism. Taking the user requests in cluster one as an example, it is as follows:

[0156] S2-1. Introduce an M / M / 1 queue to represent the UAV service process and capture the additional delay load brought by cache refreshing. The average refresh probability of the files at the UAV end within the time slot can be expressed as:

[0157]

[0158] wherein, is the probability that a request for a file initiated by a user in Cluster 1 triggers cache refresh at the UAV end during time slot ; is the request rate of a single file at the UAV end by users in Cluster 1, ; is the total request rate of all users in Cluster 1, ; is the popularity of file during time slot ; represents the file cache set of the UAV within time slot ; is the base of the natural logarithm, ; is the file refresh rate from the source node to the UAV

[0159] The average age of information of the file received by users in Cluster 1 from the UAV end during time slot is:

[0160]

[0161] wherein, is the average service rate from the UAV end to all users in Cluster 1, is the file refresh rate from the source node to the UAV end, is the cache refresh window size, is the request rate of a single file at the UAV end by users in Cluster 1, ;

[0162] The average service delay from the UAV end to users in Cluster 1 can be expressed as:

[0163]

[0164] wherein, is the average refresh probability of the file requested by users in Cluster 1 at the UAV end during time slot is the UAV The average service rate to all users in Cluster 1 is the file refresh rate from the source node to the UAV The total request rate of users in Cluster 1 for the cached files on the UAV side is the total request rate of all users in Cluster 1 is the file in time slot popularity represents the file cache set of the UAV in time slot

[0165] The average probability of cache refresh triggered by file requests initiated by users in Cluster 1 at the UAV side can be expressed as:

[0166]

[0167] In the formula, is the probability that a request for a file initiated by users in Cluster 1 at the UAV side in time slot triggers cache refresh, is the request rate of users in Cluster 1 for a single file at the UAV side is the total request rate of all users in Cluster 1 is the file in time slot popularity represents the file cache set of the UAV in time slot is the cache refresh window size, is the base of the natural logarithm, is the file refresh rate from the source node to the UAV

[0168] The average age of information received by users in Cluster 1 from the files at the UAV side is:

[0169]

[0170] In the formula, is the average service rate from the UAV to all users in Cluster 1, is the UAV and the UAV ​​​​​​​​​The transmission rate between is the file refresh rate from the source node to the UAV and is the cache refresh window size. is the request rate of a single file from the users in Cluster 1 to the UAV side, where is the base of the natural logarithm;

[0171] The average service delay from the UAV side to the users in Cluster 1 can be expressed as:

[0172]

[0173] In the formula, is the average refresh probability of the file requested by the Cluster 1 users at the UAV side within the time slot , is the average service rate from the UAV side to all users in Cluster 1, is the transmission rate between the UAV and the UAV , is the file refresh rate from the source node to the UAV , is the total request rate of the users in Cluster 1 for the cached files on the UAV side, is the total request rate of all users in Cluster 1, is the popularity of the file within the time slot , represents the file cache set of the UAV within the time slot .

[0174] S2-2 introduces an M / M / 1 queue to represent the base station service process and capture the additional delay load brought by cache refresh. At the base station side, the average probability that a file request initiated by a Cluster 1 user triggers cache refresh can be expressed as:

[0175]

[0176] In the formula, is the probability that a request for a file initiated by a Cluster 1 user at the base station side triggers cache refresh during the time slot , is the request rate of a single file from the users in Cluster 1 to the base station side, where is the total request rate of all users in Cluster 1, is the file at time slot popularity, represents the file cache set of the base station at time slot inside, is the cache refresh window size, is the base of the natural logarithm, is the file refresh rate from the source node to the base station ;

[0177] The average age of information of the file received by the users in Cluster 1 from the base station at time slot is: For the equation,

[0178]

[0179] where, is the average service rate of the base station to all users in Cluster 1, is the file refresh rate from the source node to the base station ; is the cache refresh window size, is the request rate of a single file at the base station by the users in Cluster 1, is the base of the natural logarithm;

[0180] The average service delay of the base station to the users in Cluster 1 can be expressed as:

[0181]

[0182] where, is the average refresh probability of the file at the base station requested by the users in Cluster 1 at time slot is the base station to all users in Cluster 1 average service rate, is the file refresh rate from the source node to the base station ; is the total request rate of the users in Cluster 1 for the cached files on the base station side, is the total request rate of all users in Cluster 1, is the file at time slot popularity, represents the base station at time slot The file cache set within

[0183] Users in cluster S2-3 can be analyzed in the same way. Users in cluster two receive the UAV within the time slot , the UAV , and the base station The average age of information of the end file are respectively:

[0184]

[0185]

[0186]

[0187] In the formula, is the average transmission rate from the UAV to the users in cluster two, is the transmission rate between the UAV and the UAV , is the average transmission rate from the base station to the users in cluster two, , and are respectively the file refresh rates from the source node to the UAV , the UAV , and the base station , is the cache refresh window, , and are respectively the request rates of the users in cluster two for a single file at the UAV end, the UAV end, and the base station end, is the total request rate of the users in cluster two, is the file at the time slot popularity, , and are respectively the file cache sets of the UAV , the UAV , and the base station within the time slot ;

[0188] The average service delay from the UAV , the UAV , and the base station to the users in cluster two can be expressed as:

[0189]

[0190]

[0191]

[0192] Wherein, 、 and are the average probabilities of cache refresh triggered by file requests initiated by users in cluster two at the UAV , UAV and base station respectively; 、 and are the probabilities of cache refresh triggered by a request for a file initiated by a user in cluster two at the UAV , UAV and base station in time slot respectively; 、 and are the request rates of a single file at the UAV end, UAV end and base station end by users in cluster two respectively, is the total request rate of users in cluster two, is the popularity of file in time slot , 、 and are the file cache sets of the UAV , UAV and base station within time slot respectively; 、 and are the total request rates of cached files at the UAV , UAV and base station ends by users in cluster two respectively, is the average transmission rate from the UAV to users in cluster two, is the transmission rate between the UAV and the UAV , is the average transmission rate from the base station to users in cluster two, 、 and are the file refresh rates from the source node to the UAV , the UAV and the base station , respectively;

[0193] S2-4 The average service delay of the entire system (wireless caching network system) in time slot can be expressed as:

[0194] ;

[0195] S3. Determine the UAV deployment location based on the weighted K-means algorithm considering user location, user activity, and user importance to improve the overall user experience, as follows:

[0196] S3-1 Establish decision variables: Define the locations of UAV and UAV as and , and a set of decision variables represents the caching location of file in time slot :

[0197]

[0198] where represents UAV , represents UAV , represents the base station ; is defined as the caching location of each file in time slot , is the total number of files;

[0199] S3-2 Establish the optimization objective. The objective of the optimization process is to determine the UAV deployment location and find a set of file caching decisions under various constraints to minimize the delay of the wireless caching network system. The specific objective function is defined as the long-term system delay:

[0200]

[0201] In the formula, is the average service delay in time slot , is the expectation operator, indicating taking the expected value of the random variable in the parentheses, is the time step, represents the limit operation;

[0202] S3-3 sets the constraint conditions for the delay optimization decision, specifically:

[0203] UAV cache capacity constraint: The cache capacity of the UAV is limited, and the files cached on the UAV cannot exceed its maximum cache number , which is expressed by the formula:

[0204]

[0205] In the formula, is the unified size of all files, represents the file cached on the UAV in time slot , is the total number of files;

[0206] Cache decision constraint: Each file can only be cached on one node, which is expressed by the formula:

[0207]

[0208] File content freshness constraint: The users in Cluster 1 and Cluster 2 have timeliness requirements for the obtained files. The files obtained from the UAV or the base station cannot exceed the maximum limit of the file information age when reaching the user side , which is expressed by the formula:

[0209]

[0210]

[0211] In the formula, , are the request rates of the users in Cluster for the files cached on the UAV side and the base station side respectively, , are the file cache sets of the UAV and the base station in time slot respectively, , are the request rates of the users in Cluster for a single file cached on the UAV side and the base station side respectively, , are the files received by the users in Cluster from the UAV side and the base station side in time slot respectively Average information age;

[0212] The stability requirement of the wireless caching network system: In the adopted M / M / 1 service system, the file request arrival rate at the UAV and the base station is less than the file request departure rate to ensure system stability. The formula is expressed as:

[0213]

[0214] In the formula, and are respectively the request rates of the users in cluster for the cached files at the UAV side and the base station side. and are respectively the average probabilities of cache refreshing triggered by the file requests initiated by the users in cluster at the UAV side and the base station side. and are respectively the average transmission rates from the UAV and the base station to the users in cluster . is the transmission rate between the UAV and the UAV . and are respectively the file refreshing rates from the source node to the UAV and the base station .

[0215] The optimization problem and its constraints are expressed in the following form:

[0216]

[0217] In the formula, and are respectively the positions of the UAV and the UAV . is the cache position of each file in time slot . indicates that file is cached at node in time slot . represents the UAV . represents the UAV . represents the base station . is the long-term system delay; C1 indicates that the cache capacity of the UAV is limited, and the files cached on the UAV cannot exceed its maximum cache number , is the unified size of all files, indicates that in time slot file is cached on the UAV , is the total number of files; C2 indicates that each file can only be cached on one node; C3 and C4 indicate that the files obtained from the UAV or the base station cannot exceed the maximum age limit of file information when reaching the user side ; , are respectively the request rates of the users in the cluster for the files cached at the UAV side and the base station side, , are respectively the file cache sets of the UAV and the base station in time slot , , are respectively the request rates of the users in the cluster for a single file at the UAV side and the base station side, , are respectively the average ages of the files received by the users in the cluster in time slot from the UAV side and the base station side ; C5 and C6 indicate that the file request arrival rates at the UAV and the base station side must be less than the file request departure rates; , are respectively the request rates of the users in the cluster for the files cached at the UAV side and the base station side, , are respectively the average probabilities of triggering cache refresh when the file requests initiated by the users in the cluster at the UAV and the base station side, , are respectively the UAV and the base station to the cluster The average transmission rate of users in is the transmission rate between the UAV and the UAV ; , are respectively the file refresh rates from the source node to the UAV and the base station ;

[0218] S3-4 splits the optimization problem in S3-3 into two sub-problems, including: the UAV position deployment problem and the file caching decision problem; uses the weighted K-means algorithm based on user location, user activity, and user importance to determine the UAV deployment position, specifically:

[0219] Initialization: Randomly generate the initial position of the UAV ;

[0220] Calculate the distance from each user to the UAV :

[0221]

[0222] In the formula, is the position of user ; is the total set of users, is the total number of users;

[0223] Assign user to the nearest UAV , where ;

[0224] Recalculate the position of the UAV based on user location, user activity, and user importance:

[0225]

[0226] In the formula, is the position of user ; is the user activity, indicating the probability that user sends a request; is the user importance, indicating the priority of the user's request; is the total set of users in cluster , is the total number of users in cluster ;

[0227] Termination condition: Continuously iterate the above processes of initialization, calculating the positions of the UAVs, etc., until the termination condition is met (the sum of the position changes of the two UAVs in two consecutive times is less than the threshold );

[0228] S4. Use the deep reinforcement learning algorithm to dynamically adjust the cache placement strategy to obtain the optimal cache strategy, specifically as follows:

[0229] Use the deep Q-network to solve the file caching decision problem. The deep Q-network algorithm includes the following algorithm steps:

[0230] S4-1 Initialization: Initialize the state space, action space, and target reward function of the DQN, and randomly initialize the parameters of the Q-network , and create a target Q-network;

[0231] S4-2 State definition: The state of the wireless cache network system consists of the file request probability vector in the current time slot, the remaining battery power of the UAV in the current time slot and the cache strategy in the previous time slot ;

[0232] S4-3 Action selection: The action represents the cache strategy in the current time slot, , and decides which files to cache in the UAV or the base station in the time slot ;

[0233] S4-4 Reward calculation: The reward function is defined according to the optimization objective and constraint conditions, including the minimization of the system average delay , and by introducing a penalty term to ensure the satisfaction of the constraint conditions, specifically the cache capacity limit, cache location limit, file freshness requirement, and system stability constraint;

[0234] The expression of the reward function is:

[0235]

[0236] where is the average service delay of the system in the time slot , represents the UAV cache capacity constraint to ensure that the files cached on the UAV do not exceed its maximum cache number , represents the cache decision constraint to ensure that each file can only be cached at one node, represents the UAV Terminal file content freshness constraint to ensure that the file obtained from the UAV does not exceed the maximum bound of the file information age when it reaches the user terminal. , denotes the base station Terminal file content freshness constraint to ensure that the file obtained from the base station does not exceed the maximum bound of the file information age when it reaches the user terminal. , denotes the UAV Terminal system stability constraint to ensure that the file request arrival rate at the UAV is less than the file request departure rate. denotes the base station Terminal system stability constraint to ensure that the file request arrival rate at the base station is less than the file request departure rate. is the penalty coefficient, and the max function represents taking the maximum value in the parentheses. is the unified size of all files. is the UAV cache capacity. denotes the cache position of file in time slot . is the total number of files. , are respectively the request arrival rates of the files cached at the UAV terminal and the base station terminal by the users in the cluster . , are respectively the file cache sets of the UAV and the base station in time slot . , are respectively the request rates of a single file cached at the UAV terminal and the base station terminal by the users in the cluster . , are respectively the average information ages of the files received by the users from the UAV terminal and the base station terminal in the cluster and the base station terminal in time slot . is the maximum bound of the file information age. , are respectively at the UAV and the base station terminals, in the cluster The average probability of cache refresh triggered by user-initiated file requests, and are the average transmission rates of the user in the cluster from the drone and the base station to the cluster respectively; is the transmission rate between the drone and the drone ; and are the file refresh rates from the source node to the drone and the base station respectively;

[0237] S4-6 Network Update: According to the experience replay mechanism, a small batch of sample sets are randomly selected for training, and the parameters of the Q network are updated using the target Q value ; ;

[0238] S4-7 Action Execution: Execute the selected action , update the system state , and record the reward ;

[0239] S4-8 Termination Condition: Continuously iterate the above S4-1 to S4-7 processes until the termination condition (Q value convergence) is met;

[0240] After completing the optimization solution, when using the trained DQN to obtain the file caching strategy, by inputting the current system state into the trained Q network, calculate the Q values of each possible action, and select the action with the largest Q value , and the selected action is the optimal caching strategy for the current time slot. According to the optimal caching strategy, dynamically adjust the caching decisions of the drone and the base station to optimize the system performance.

[0241] Next, the technical solution of the present invention will be further elaborated in combination with specific embodiments:

[0242] An embodiment of the present invention provides a method for optimizing the delay of a multi-drone assisted wireless caching network. To verify the beneficial effects of the present invention, scientific demonstrations are carried out through simulation experiments.

[0243] Figure 3 The delay comparison graph of the DQN algorithm proposed in this embodiment with the traditional random algorithm and the particle swarm optimization algorithm within 1000 test rounds is given. It can be seen that the delay of the DQN algorithm proposed in this embodiment is better than that of the particle swarm optimization algorithm and the random algorithm.

[0244] In terms of computational complexity, the simulation platform is the CPU Intel i5-13600. During the test phase, the trained DQN model was compared with other optimization algorithms in terms of time complexity. The running time of each algorithm per round is shown in Table 1:

[0245] Table 1 Average duration per round (seconds)

[0246]

[0247] The trained DQN model of the present invention is 487 times faster than the particle swarm optimization algorithm. Combining Figure 3 with Table 1, the DQN algorithm proposed in the present invention can not only effectively reduce the system delay, but also has extremely low time complexity, fully proving the effectiveness and practicability of the method.

[0248] Embodiment 2:

[0249] This embodiment provides a multi-UAV-assisted wireless caching network delay optimization system for implementing the multi-UAV-assisted wireless caching network delay optimization method described in Embodiment 1, including:

[0250] The first construction module is used to construct a wireless caching network architecture model for the collaborative work of multiple UAVs and a ground base station based on the remaining battery ratio of the UAVs, so as to improve the file distribution efficiency and user response speed;

[0251] The update mechanism improvement module is used to introduce the age of information as a key indicator of file content freshness based on the wireless caching network architecture model to improve the caching update mechanism;

[0252] The second construction module determines the UAV deployment location based on the weighted K-means algorithm of user location, user activity and user importance to improve the overall user experience;

[0253] The third construction module is used to construct an intelligent optimization model and dynamically adjust the caching placement strategy using a deep reinforcement learning algorithm to obtain the optimal caching strategy.

[0254] Specifically, the above-mentioned first construction module, update mechanism improvement module, second construction module and third construction module can be embedded in a computer processing system. The computer calls the above-mentioned modules to complete the task of delay optimization according to the multi-UAV-assisted wireless caching network delay optimization method provided above; the above-mentioned first construction module, update mechanism improvement module, second construction module and third construction module can perform operations according to the specific steps given by the multi-UAV-assisted wireless caching network delay optimization method.

[0255] It should be noted that it should be understood that the division of each module of the above system is only a division of logical functions. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; it is also possible that some modules are implemented in the form of software called by a processing element, and some modules are implemented in the form of hardware. For example, the first construction module can be a separately established processing element, or can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and the function of the above signal processing module is called and executed by a certain processing element of the above device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. The processing element mentioned here can be an integrated circuit with the ability to process signals. In the implementation process, each step of the above method or each of the above modules can be completed by the integrated logic circuit of the hardware in the processor element or the instruction in the form of software.

[0256] For example, the above-mentioned modules can be one or more integrated circuits configured to implement the above method. For example: one or more Application Specific Integrated Circuits (ASICs), or, one or more Digital Signal Processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0257] Embodiment 3:

[0258] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The computer program capable of running on the processor is stored in the memory. When the processor loads and executes the computer program, the above-mentioned multi-UAV-assisted wireless caching network delay optimization method is adopted.

[0259] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server. Moreover, the terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may further include input / output devices, network access devices, and a bus, etc.

[0260] Furthermore, the processor can be a central processing unit (CPU). Of course, according to the actual usage situation, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be used. The general-purpose processor can be a microprocessor or any conventional processor, etc. This application does not make any restrictions on this.

[0261] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article, or device.

[0262] For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. When an element is referred to as being "assembled on", "installed on", "fixed to", or "disposed on" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "upper", "lower", "left", "right", and similar expressions used herein are only for the purpose of illustration and do not represent the only implementation manner.

[0263] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

[0264] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

Claims

1. A method for optimizing the delay of a multi-drone-assisted wireless cache network, wherein the multi-drone-assisted wireless cache network includes users, base stations, and drones that respond to users according to user requests in a cellular network, and is characterized in that: The following steps are involved: Users in the cellular network area are divided into multiple clusters, each of which corresponds to a drone for service. The drones are equipped with a cache unit, which is used to cache the file content published by the ground source node. The drone corresponding to the user cluster transmits the cached file content to the user through the channel according to the user's request; If the file content is located in the cache unit of the drone in the user's adjacent cluster, the cache unit of the drone in the adjacent cluster is transmitted to the cache unit of the drone in the user's corresponding cluster, and the drone in the user's corresponding cluster transmits the file to the user through the channel; Taking the cache capacity of drones, the cache location of file contents, the freshness requirements of file contents and the stability of multi-drone-assisted wireless cache network as constraints, a deep reinforcement learning algorithm is used for optimization. The network delay of multi-drone-assisted wireless cache is optimized with the goal of maximizing the response rate of multi-drone-assisted wireless cache network to users according to the file content according to user requests. Taking the cache capacity of drones, the cache location of file contents, the freshness requirements of file contents, and the stability of multi-drone-assisted wireless cache networks as constraints, a deep reinforcement learning algorithm is used for optimization, as follows: Step (81) Establish decision variables: define drone and drones The locations are and , a set of decision variables Indicates time slot document Cache location: in, When the cache node is a drone , When the cache node is a drone , When the cache node is a base station ; Defined as time slot The cache location of each file, is the total number of files; Step (82) establishes the optimization goal. The goal of the optimization process is to determine the deployment location of the UAV and find a set of file cache decisions under various constraints to minimize the system delay. The specific objective function is defined as the long-term system delay: In the formula, It is in the time slot The average service delay of is the expectation operator, which means taking the expected value of the random variable in the brackets. is the time step, Indicates extreme operation; Step (83) sets the constraints for the delay optimization decision, specifically: Step (831) UAV cache capacity constraint: The cache capacity of the UAV is limited. The number of files on the cache cannot exceed its maximum cache size. , the formula is: In the formula, is the uniform size of all files, Indicates time slot document Cache on drone , is the total number of files; Step (832) Cache decision constraint: Each file can only be cached in one node, expressed as: Step (833) File content freshness constraint: Users in cluster 1 and cluster 2 have timeliness requirements for the files they obtain. The files obtained from the drone or base station cannot exceed the maximum limit of file information age when they reach the user end. , the formula is: In the formula, , Clusters Chinese users use drones Terminal, Base Station The request rate of the client cache file, , UAVs , Base Station In time slot The file cache collection within, , Clusters Chinese users use drones Terminal, Base Station Single file The request rate, , Clusters The user in the time slot Receive drone Terminal, Base Station Terminal File Average information age of Step (834) System stability requirement: The file request arrival rate between the drone and the base station is less than the file request departure rate to ensure system stability. The formula is: In the formula, , Clusters Chinese users use drones Terminal, Base Station The request rate of the client cache file, , In drones , Base Station End, Cluster Average probability of a file request initiated by a user triggering a cache refresh, , UAVs , Base Station To Cluster The average transmission rate of users in It's a drone and drones The transmission rate between , From source node to drone , Base Station The file refresh rate; Step (835) The optimization problem and its constraints are expressed as follows: 。 2. The multi-UAV assisted wireless cache network delay optimization method according to claim 1 is characterized in that: The users in the cellular network area are divided into two clusters, cluster one and cluster two, and each cluster is equipped with a drone and drones The base station acts as a backup transmission node and can provide file transfer services to remote users when the drone cache is insufficient. In addition, a source node exists in the cellular network for real-time monitoring and generating the latest file version; , drones , drones , and the base station caches the dynamic file content generated by the source node through wireless transmission, and sends the file to the user on demand.

3. The multi-UAV assisted wireless cache network delay optimization method according to claim 2 is characterized in that: Drones and drones The current time slot will be based on its own power , The cache quantity of the files cached on the drone side is allocated in the following proportion: .

4. The multi-UAV assisted wireless cache network delay optimization method according to claim 3 is characterized in that: The file content is transmitted to the user through the channel by the drone of the user's corresponding cluster according to the user's request, including the following situations: (41) If the file content requested by the user in cluster 1 is When caching on the client, the drone Provided to users through a single channel; (42) If the file content requested by the user in cluster 1 is on the drone The end cache is handled by the drone Send to drone , and then by drone Send to users; (43) If the file content requested by the user in cluster 1 is cached at the base station, it will be provided to the user through a single channel by the base station.

5. The multi-UAV assisted wireless cache network delay optimization method according to claim 4 is characterized in that: The channel between the user and the drone is modeled as a line-of-sight or non-line-of-sight air-to-ground connection. The average service rate to users within its service range is expressed as: In the formula, It's a drone To User The service rate, is the user's activity, indicating that the user Probability of sending a request; is the user importance, indicating the priority of the user sending the request; It's a drone Available bandwidth, is the transmitting power of the drone, It's a drone With users The channel gain between Is a single file of uniform size, is Gaussian noise, It's a drone The number of users in the service range, in the scenario of two adjacent drones, ,in I 1 and I 2 respectively represent drones and drones The number of users in the service area, I Indicates two adjacent drones and drones The total number of users in the service area; The transmission link between the source node and the UAV also considers the air-to-ground connection model of line-of-sight or non-line-of-sight connection. The file refresh rate is expressed as: In the formula, , From source node to drone The transmission bandwidth and transmission power, The source node and the drone The channel gain between Is a single file of uniform size, is Gaussian noise; Drones and drones Considering the air-to-air connection model with line-of-sight connection, the transmission rate between two UAVs is expressed as: In the formula, is the transmission bandwidth between the two drones, is the transmission power between the two drones, is the additional attenuation factor for the line-of-sight link, and UAVs and drones location, Is a single file of uniform size, is Gaussian noise; The channel between the user and the base station is a ground-to-ground connection, based on path loss and Rayleigh fading modeling, the base station to cluster The average service rate of users in is expressed as: In the formula, Base station to user The service rate, is the user's activity, indicating that the user Probability of sending a request; is the user importance, indicating the priority of the user sending the request; is the available bandwidth of the base station, is the transmission power of the base station, Base station and user The channel gain between Is a single file of uniform size, is Gaussian noise, It's a drone The number of users in the service area; The transmission link between the source node and the base station is also modeled based on path loss and Rayleigh fading. The file refresh rate from the source node to the base station is: In the formula, , are the transmission bandwidth and transmission power from the source node to the base station, respectively. is the channel gain between the source node and the base station, Is a single file of uniform size, is Gaussian noise.

6. The multi-UAV assisted wireless cache network delay optimization method according to claim 5 is characterized in that: Before sending a file, the drone or base station will check the freshness of the file and introduce the information age as an indicator to measure the freshness of the file content. Take the user request in cluster 1 as an example, the details are as follows: Step (61) introduces an M / M / 1 queue to represent the drone service process and capture the additional latency load caused by cache refresh. End file in time slot The internal average refresh probability is expressed as: In the formula, It's a drone End in time slot In the cluster, a user initiates a request to a file The probability that a request triggers a cache refresh, The user in cluster 1 has a Single file The request rate, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates drone In time slot The file cache collection within, is the cache refresh window size, is the base of natural logarithms, From source node to drone The file refresh rate; The user in cluster 1 is in time slot Receive drone Terminal File The average information age is: In the formula, It's a drone The average service rate to all users in cluster 1, From source node to drone The file refresh rate, is the cache refresh window size, The user in cluster 1 has a Single file The request rate, is the base of natural logarithms; Drones The average service delay from the end to the user in cluster 1 is expressed as: In the formula, is the drone requested by a cluster user End file in time slot Average refresh probability, It's a drone The average service rate to all users in cluster 1, From source node to drone The file refresh rate, The user in cluster 1 has a The total request rate of the side cache files, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates drone In time slot The file cache collection within; Drones The average probability of a file request initiated by a user in cluster 1 triggering a cache refresh is expressed as: In the formula, It's a drone End in time slot In the cluster, a user initiates a request to a file The probability that a request triggers a cache refresh, The user in cluster 1 has a Single file The request rate, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates drone In time slot The file cache collection within, is the cache refresh window size, is the base of natural logarithms, From source node to drone The file refresh rate; Users in cluster 1 receive the drone Terminal File The average information age is: In the formula, It's a drone The average service rate to all users in cluster 1, It's a drone and drones The transmission rate between From source node to drone The file refresh rate, is the cache refresh window size, The user in cluster 1 has a Single file The request rate, is the base of natural logarithms; Drones The average service delay from the end to the user in cluster 1 is expressed as: In the formula, is the drone requested by a cluster user End file in time slot Average refresh probability, It's a drone The average service rate to all users in cluster 1, It's a drone and drones The transmission rate between From source node to drone The file refresh rate, The user in cluster 1 has a The total request rate of the side cache files, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates drone In time slot The file cache collection within; Step (62) introduces an M / M / 1 queue to represent the base station service process and capture the additional delay load caused by cache refresh. The average probability of a file request initiated by a user in cluster 1 triggering a cache refresh is expressed as: In the formula, It is a base station End in time slot In the cluster, a user initiates a request to a file The probability that a request triggers a cache refresh, is the user in cluster 1 to the base station Single file The request rate, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates base station In time slot The file cache collection within, is the cache refresh window size, is the base of natural logarithms, From source node to base station The file refresh rate; The user in cluster 1 is in time slot Received from base station Terminal File The average information age is: In the formula, It is a base station The average service rate to all users in cluster 1, From source node to base station The file refresh rate, is the cache refresh window size, is the user in cluster 1 to the base station Single file The request rate, is the base of natural logarithms; Base Station The average service delay to users in cluster 1 is expressed as: In the formula, is the base station requested by cluster 1 user End file in time slot Average refresh probability, It is a base station The average service rate to all users in cluster 1, From source node to base station The file refresh rate, is the user in cluster 1 to the base station The total request rate of the side cache files, is the total request rate of all users in cluster 1, is a file In time slot The popularity of Indicates base station In time slot A collection of file caches within.

7. The multi-UAV assisted wireless cache network delay optimization method according to claim 6 is characterized in that: The users in cluster 2 are analyzed in the same way as in step (61) and step (62). The users in cluster 2 are in time slot Receive drone , drones and base station Terminal File The average information ages are: In the formula, It's a drone The average transmission rate to users in cluster 2, It's a drone and drones The transmission rate between It is a base station The average transmission rate to users in cluster 2, , and From source node to drone , drones and base station The file refresh rate, is the cache refresh window, , and They are respectively the users in cluster 2 and the drones Terminal, drone Terminal and base station Single file The request rate, is the total request rate of users in cluster 2, is a file In time slot The popularity of , and UAVs , drones and base station In time slot The file cache collection within; Drones , drones and base station The average service delay to users in cluster 2 is expressed as: In the formula, , and In drones , drones and base station On the other end, the average probability of a file request initiated by a user in cluster 2 triggering a cache refresh; , and In drones , drones and base station The user in cluster 2 is in time slot A pair of documents The probability that a request triggers a cache refresh; , and They are respectively the users in cluster 2 and the drones Terminal, drone Terminal and base station Single file The request rate, is the total request rate of users in cluster 2, is a file In time slot The popularity of , and UAVs , drones and base station In time slot The file cache collection within; , and They are respectively the users in cluster 2 and the drones , drones and base station The total request rate of the client cache files, It's a drone The average transmission rate to users in cluster 2, It's a drone and drones The transmission rate between It is a base station The average transmission rate to users in cluster 2, , and From source node to drone , drones and base station The file refresh rate; The wireless cache network as a whole is in the time slot The average service delay is expressed as: 。 8. The multi-UAV assisted wireless cache network delay optimization method according to claim 7 is characterized in that: The optimization problem in step (835) is divided into two sub-problems, including: the drone location deployment problem and the file cache decision problem, as follows: Step (91) uses a weighted K-means algorithm based on user location, user activity, and user importance to determine the drone deployment location, specifically: Step (91.1) Initialization: Randomly generate drones Initial position ; Step (91.2) calculates for each user To drones Distance: In the formula, Is a user location, is the total set of users, is the total number of users; Step (91.3) will user Assign to the nearest drone ,in ; Step (91.4) recalculates the drone based on the user's location, user activity and user importance Location: In the formula, Is a user location; is the user's activity, indicating that the user Probability of sending a request; is the user importance, indicating the priority of the user sending the request; It is a cluster The total set of users in It is a cluster Total number of users in Step (91.5) Termination condition: Continue to iterate steps (91.1) to (91.4) until the drone meets and drones The sum of the two position changes is less than the threshold ; Step (92) uses a deep Q network to solve the file cache decision problem. The deep Q network algorithm includes the following algorithm steps: Step (92.1) Initialization: Initialize the state space, action space, and target reward function of the deep Q network, and randomly initialize the parameters of the Q network , and create the target Q network; Step (92.2) State definition: System state The probability vector of file request in the current time slot 、The remaining battery power of the drone in the current time slot and the previous time slot cache strategy Composition, of which is a file In time slot The popularity of F Indicates the total number of files; Step (92.3) Action selection: Action Indicates the cache strategy for the current time slot, , decide on the time slot Cache to drone or base station files, Indicates time slot document The cache location of Step (92.4) Reward calculation: Reward function Defined according to optimization objectives and constraints, including system average delay The minimization of the resource consumption is carried out, and the penalty term is introduced to ensure that the constraints are met, which are the cache capacity limit, cache location limit, file freshness requirement and system stability constraint. The reward function expression is: in, Is the system in time slot Average service delay of Represents the drone cache capacity constraint, ensuring that the cache is in the drone The maximum number of files in the cache cannot exceed ; Represents the cache decision constraint, ensuring that each file can only be cached on one node; Indicates drone The freshness of the end file content is constrained to ensure that The file obtained by the client cannot exceed the maximum age limit of the file information when it reaches the user end. ; Indicates base station The freshness of the end file content is constrained to ensure that The file obtained by the client cannot exceed the maximum age limit of the file information when it reaches the user end. ; Indicates drone End system stability constraints ensure that drones The file request arrival rate at the client is less than the file request departure rate; Indicates base station End system stability constraints ensure base station The file request arrival rate at the client is less than the file request departure rate. is the penalty coefficient, and the max function means taking the maximum value in the brackets. is the uniform size of all files, It's a drone Cache capacity, Indicates time slot document The cache location, is the total number of files, , Clusters Chinese users use drones Terminal, Base Station The request arrival rate of the client cache file, , UAVs , Base Station In time slot The file cache collection within, , Clusters Chinese users use drones Terminal, Base Station Single file The request rate, , Clusters The user in the time slot Receive drone Terminal, Base Station Terminal File The average information age of Is the maximum age limit of file information, , In drones , Base Station End, Cluster Average probability of a file request initiated by a user triggering a cache refresh, , UAVs , Base Station To Cluster The average transmission rate of users in It's a drone and drones The transmission rate between , From source node to drone , Base Station The file refresh rate; Step (92.5) Network update: According to the experience replay mechanism, randomly select a small batch of samples Perform training using the target Q value Update the parameters of the Q network ,in, is a discount factor used to weigh the importance of current rewards and future rewards, Indicates in status Next, one of all the actions; Step (92.6) Action Execution: Execute the selected action , update system status , record reward ; Step (92.7) termination condition: continue to iterate step (92.1) to step (92.6) until the termination condition of Q value convergence is met; After the optimization solution is completed in step (92.8), when using the trained DQN to obtain the file cache strategy, the current system state is input into the trained Q network, the Q value of each action is calculated, and the action with the largest Q value is selected. ,in, Indicates in status Next, one of all possible actions; the action chosen This is the optimal cache strategy for the current time slot.

9. A multi-UAV assisted wireless cache network delay optimization system, used to implement the multi-UAV assisted wireless cache network delay optimization method according to any one of claims 1 to 8, characterized in that: include: The first building module is used to build a wireless cache network architecture model based on the remaining power ratio of the drones and the ground base station to work together, so as to improve the file distribution efficiency and user response speed; The update mechanism improvement module is used to introduce information age as a key indicator of file content freshness based on the wireless cache network architecture model and improve the cache update mechanism; The second building block determines the drone deployment location based on the weighted K-means algorithm based on user location, user activity and user importance to improve the overall user experience; The third building module is used to build an intelligent optimization model, which uses a deep reinforcement learning algorithm to dynamically adjust the cache placement strategy to obtain the optimal cache strategy.

Images