Unmanned aerial vehicle virtual reality system and implementation method thereof

Abstract

The invention discloses an unmanned aerial vehicle virtual reality system and an implementation method thereof, and relates to the field of virtual reality, and the system comprises an unmanned aerial vehicle, an MEC server, a block chain and a user VR device. The MEC server and the block chain are both integrated in the unmanned aerial vehicle; performing data interaction between the block chain and the MEC server; the unmanned aerial vehicle and the user VR equipment carry out rendering video transmission; the user VR equipment is used for receiving the rendered video; the block chain is used for storing position information of the unmanned aerial vehicle and resource allocation information of the unmanned aerial vehicle and the user VR equipment; a transmission model and a rendering model are implanted into the MEC server, and a problem model is constructed; and the MEC server is used for generating and transmitting a rendered video by adopting the transmission model, the rendering model and the problem model. According to the application, the security and privacy can be enhanced, the optimal balance between the energy consumption and the rendering delay is realized, and thus the performance of the block chain and the virtual reality system can be ensured.

Description

Technical Field

[0001] This application relates to the field of virtual reality, and particularly to a drone virtual reality system and an implementation method thereof. Background Art

[0002] With the rapid development of intelligent devices and computing technologies, the scale and diversity of virtual reality are constantly expanding. The immersive experience provided by virtual reality (VR) applications is considered one of its main attractions, achieved by leveraging high-quality resolution, panoramic scenes, and advanced audio. This combination provides users with unparalleled sensory stimulation, enhancing their participation and enjoyment in the virtual reality environment. Currently, virtual reality has become one of the most popular applications in both the industrial and academic fields. However, different from traditional applications, supporting virtual reality applications requires a large amount of computing and bandwidth resources, which poses a huge pressure on the current wireless network. Transmitting VR videos requires five times the bandwidth required for conventional videos. In addition, the latency of motion-to-photon (MTP) should not exceed 20 milliseconds to prevent dizziness.

[0003] To meet the high service requirements, multi-access edge computing (MEC) has been regarded as a mature solution in recent years. By offloading computationally intensive tasks to the MEC side, the rendering latency can be significantly reduced. However, traditional edge computing has certain limitations in terms of coverage and transmission efficiency. On the one hand, MEC servers are usually deployed in fixed traditional base-stations (BSs) and may not be able to provide their advantages in special situations such as infrastructure damage or lack of available terrestrial networks. On the other hand, the transmission links of terrestrial MEC systems are usually dominated by non-line-of-sight (NLoS). Due to the poor channel conditions, the transmission rate is usually greatly limited.

[0004] The integration of unmanned aerial vehicles (UAVs, i.e., drones) and MEC brings new opportunities to address the challenges faced by traditional MEC systems. The upgraded system can provide high-speed communication and computing services for virtual reality applications. By leveraging the aerial hovering ability of drones, the probability of line-of-sight (LoS) links can be significantly increased, enhancing the delivery efficiency of VR content. In addition, drones can be deployed more flexibly and easily, pushing the computing resources closer to the user side, thereby reducing the MTP latency. The drone-supported MEC system marks a new era of modern wireless virtual reality applications, thus providing higher efficiency, reliability, and flexibility.

[0005] However, there are still some challenges in the drone - supported MEC system. First, the rendering process of virtual reality applications requires a large amount of computing resources, which may lead to an energy crisis because the energy capacity of drones is limited. Reducing computing resources is an effective way to reduce energy consumption, but it may also have a significant impact on the MTP latency. Therefore, how to achieve the optimal balance between MTP latency and energy consumption is a challenge. Second, the drone - supported MEC system faces huge security risks, especially in an untrusted environment with malicious attackers. Specifically, malicious nodes may tamper with resource allocation and rendering decisions, resulting in a decrease in resource efficiency and the quality of experience (QoE) of users. The blockchain - based MEC framework is regarded as a promising solution, under which important information can be packaged into blocks in the form of transactions. Due to the immutability of the blockchain, the integrated framework can prevent fraud in a distributed manner. However, maintaining the normal operation of the blockchain also requires a large amount of computing resources. Therefore, how to ensure the performance of the blockchain and the virtual reality system is also a major challenge. Summary of the Invention

[0006] To solve the above problems existing in the prior art, the present application provides a drone virtual reality system and its implementation method.

[0007] To achieve the above object, the present application provides the following solutions: In a first aspect, the present application provides a drone virtual reality system, including: a drone, an MEC server, a blockchain, and a user VR device; Both the MEC server and the blockchain are integrated in the drone; the blockchain exchanges data with the MEC server; the drone and the user VR device perform rendering video transmission; the user VR device is used to receive the rendering video; the blockchain is used to store the location information of the drone and the resource allocation information of the drone and the user VR device; A transmission model, a rendering model are implanted in the MEC server and a problem model is constructed; the MEC server is used to generate and transmit the rendering video by using the transmission model, the rendering model, and the problem model; the transmission model is used to determine the video transmission rate between the drone and the user VR device; the rendering model is used to determine the video transmission time between the drone and the user VR device; the problem model integrates optimization objectives; the optimization objectives include: drone deployment, rendering decision, computing resource allocation, bandwidth resource allocation, and block generation decision; the block generation decision is implemented in the blockchain.

[0008] Optionally, the transmission link between the UAV and the user's VR device adopts a probabilistic LoS channel model; The process by which the MEC server determines the video transmission time and video transmission rate between the UAV and the user's VR device using the transmission model includes: Determine the distance between the UAV and the user's VR device; Based on the distance, determine the channel coefficient of the transmission link between the UAV and the user's VR device; Determine the probability of the LoS link between the UAV and the user's VR device and the probability of the NLoS link between the UAV and the user's VR device; Based on the channel coefficient, the probability of the LoS link, and the probability of the NLoS link, determine the channel power gain between the UAV and the user's VR device; Based on the channel power gain, determine the downlink transmission rate of the UAV; the downlink transmission rate is used as the video transmission rate between the UAV and the user's VR device.

[0009] Optionally, the rendering model includes a local rendering mode and an MEC rendering mode.

[0010] Optionally, in the local rendering mode, the process by which the MEC server determines the video transmission time between the UAV and the user's VR device using the rendering model includes: Based on the data size of the VR content requested by the user, the video transmission rate, and the rendering decision, determine the delay for the user's VR device to obtain the original video from the UAV; Based on the data size of the VR content requested by the user, the rendering resources of the user's VR device, the period for processing one bit of data, and the rendering decision, determine the presentation delay of the user's VR device; Based on the delay for the user's VR device to obtain the original video from the UAV and the presentation delay of the user's VR device, determine the video transmission time between the UAV and the user's VR device.

[0011] Optionally, in the MEC rendering mode, the process by which the MEC server determines the video transmission time between the UAV and the user's VR device using the rendering model includes: Based on the data size of the VR content requested by the user, the rendering resources of the user's VR device, the period for processing one bit of data, and the rendering decision, determine the rendering delay of the MEC server; Based on the data size of the VR content requested by the user, the video transmission rate, the data conversion rate, and the rendering decision, determine the time for the UAV to transmit the rendered video to the user's VR device; Determine the video transmission time between the drone and the user VR device based on the rendering delay of the MEC server and the time for the drone to transmit the rendered video to the user VR device.

[0012] Optionally, in the blockchain, an optimization algorithm based on block coordinate descent is used to determine the block producer; the block producer and the validator complete data sharing using the proof-of-stake consensus mechanism to generate the block generation decision.

[0013] Optionally, an energy consumption model is also implanted in the MEC server; the energy consumption model is used to determine the energy consumption during the entire rendering process.

[0014] Optionally, during the process of the MEC server using the transmission model, the rendering model, and the problem model to generate and transmit the rendered video, decouple the problem model into the development of the drone, the rendering decision, and the block producer selection problem, as well as the computing and bandwidth resource allocation problem; Use an algorithm based on Lagrangian multiplier iteration to solve the computing and bandwidth resource allocation problem, and use an algorithm based on deep reinforcement learning to solve the drone development, rendering decision, and block producer selection problem to obtain a combined strategy.

[0015] Optionally, the algorithm of the deep reinforcement learning is the proximal policy optimization algorithm.

[0016] In a second aspect, the present application provides an implementation method of a drone virtual reality system, including: Construct a transmission model, a rendering model, and a problem model; the problem model integrates optimization objectives; the optimization objectives include: drone deployment, rendering decision, computing resource allocation, bandwidth resource allocation, and block generation decision; the block generation decision is implemented in the blockchain; the transmission model is used to determine the video transmission rate between the drone and the user VR device; the rendering model is used to determine the video transmission time between the drone and the user VR device; Use the transmission model, the rendering model, and the problem model to generate and transmit the rendered video.

[0017] According to the specific embodiments provided by the present application, the present application has the following technical effects: The present application provides a drone virtual reality system and its implementation method. By introducing the blockchain, security and privacy can be enhanced. Moreover, by using the transmission model, the rendering model, and the problem model to implement the generation and transmission of the rendered video and complete virtual reality interaction, an optimal trade-off between energy consumption and rendering delay can be achieved, thereby ensuring the performance of the blockchain and the virtual reality system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0019] Figure 1 Schematic diagram of the architecture of a drone virtual reality system provided by an embodiment of the present application; Figure 2 Schematic diagram of the process of transmitting the rendered data between the drone and the user and receiving the original task provided by an embodiment of the present application; Figure 3 Schematic diagram of the architecture of the block coordinate descent optimization algorithm provided by another embodiment of the present application; Figure 4 Framework diagram of the proximal policy optimization algorithm provided by an embodiment of the present application; Figure 5 Schematic diagram of the performance comparison results between the solution provided by the present application and three other baseline solutions (the proposed solution without bandwidth resource optimization, the proposed solution without computing resource optimization, and the proposed solution without rendering decision optimization); Figure 6 Schematic diagram of the performance comparison results between the solution provided by the present application and four other solutions (the solution based on the deep Q-network, the solution based on the proximal policy optimization, PSwoRD, and the proposed solution for drone position optimization); Figure 7 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0021] To make the above objects, features, and advantages of the present application more clearly understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0022] The embodiments of the present application provide a drone virtual reality system, as Figure 1 shown, including: a drone, an MEC server, a blockchain, and a user VR device.

[0023] Both the MEC server and the blockchain are integrated into the drone. The blockchain interacts with the MEC server for data. The drone and the user's VR device perform rendering video transmission. The user's VR device is used to receive the rendered video. The blockchain is used to store the location information of the drone and the resource allocation information of the drone and the user's VR device.

[0024] The MEC server is implanted with a transmission model, a rendering model and constructs a problem model. The MEC server is used to generate and transmit the rendered video by using the transmission model, the rendering model and the problem model. The transmission model is used to determine the video transmission rate between the drone and the user's VR device. The rendering model is used to determine the video transmission time between the drone and the user's VR device. The problem model is integrated with optimization objectives. The optimization objectives include: drone deployment, rendering decision, computing resource allocation, bandwidth resource allocation and block generation decision. The block generation decision is implemented in the blockchain.

[0025] Based on the system architecture provided above in this application, the key decisions in the virtual reality rendering process are timely recorded in the blockchain to enhance security and privacy.

[0026] In an exemplary embodiment of this application, taking a virtual reality system supported by a drone (hanging in the air) as an example, the implementation process of the drone virtual reality system provided above in this application is described. Among them, since the MEC server is integrated into the drone, the drone has both communication and computing capabilities at the same time. The set of drones can be defined as , both represent drones. As the set of user VR devices (i.e., ground user VR devices) accessing the drone , both represent the user VR devices accessing the drone .

[0027] Assume that the drone has pre-cached the video data requested by the ground virtual reality users. In the network, each drone should provide resources to help users render virtual reality content. The VR task can be modeled by a tuple , where is the data size of the VR content requested by the user VR device , is the number of CPU (Central Processing Unit / Processor) cycles required to process one piece of data, represents the data conversion rate between the input video data and the output video data. The ground user wearing the VR device can experience immersive content by receiving the video rendered by the drone. Considering the security and privacy issues in the rendering process, the blockchain is deployed in the network to protect important decisions.

[0028] In an exemplary embodiment of the present application, the transmission link between the drone and the user's VR device adopts a probabilistic LoS channel model. Based on this, the process by which the MEC server determines the video transmission time and video transmission rate between the drone and the user's VR device includes: (1) Determine the distance between the drone and the user's VR device. For example, in this system, the drone can hover at a fixed height represented by . Assume that the horizontal coordinate of the drone is represented by . is the abscissa value of the horizontal coordinate of the drone , and is the ordinate value of the horizontal coordinate of the drone. Let be the horizontal coordinate of the user's VR device , be the abscissa value of the horizontal coordinate of the user's VR device , and be the ordinate value of the horizontal coordinate of the user's VR device . Then, the distance between the user's VR device and the drone can be given by formula (1).

[0029] (1) (2) Determine the channel coefficient of the transmission link between the drone and the user's VR device based on the distance. Among them, both LoS and NLoS links are considered in the probabilistic LoS channel model. Let represent the channel coefficient between the drone and the user's VR device , which is given by formula (2).

[0030] (2) Among them, , and respectively represent the path loss of the LoS link, the attenuation factor of the NLoS link, and the path loss exponent. The LoS link probability is mainly determined by the elevation angle and the propagation environment.

[0031] (3) Determine the probability of the LoS link between the drone and the user's VR device and the probability of the NLoS link between the drone and the user's VR device. The probability of the LoS link between the user's VR device and the drone can be expressed as formula (3). ​​​​

[0032] (3) Wherein, and represent environmental parameters.

[0033] Therefore, the probability of the NLoS link can be expressed as .

[0034] (4)Determine the channel power gain between the UAV and the user's VR device based on the channel coefficient, the probability of the LoS link, and the probability of the NLoS link. Among them, the user's VR device and the UAV The channel power gain between can be expressed as: (4) In the formula, represents the channel power gain between the user's VR device and the UAV .

[0035] (5)Determine the downlink transmission rate of the UAV based on the channel power gain. The downlink transmission rate is used as the video transmission rate between the UAV and the user's VR device.

[0036] For example, orthogonal frequency-division multiple access (OFDMA) is used to transmit video content. Each UAV occupies independent bandwidth resources to avoid interference from other UAVs in the network. Let represent the bandwidth resource allocated to the UAV . Then, the downlink transmission rate is given by formula (5).

[0037] (5) In the formula, and are the downlink transmission power and the noise power density of the UAV , respectively.

[0038] In an exemplary embodiment of the present application, a collaborative rendering method is adopted in the rendering model, which introduces two rendering modes in the system, namely, the local rendering mode and the MEC rendering mode. Among them: (1)In the local rendering mode, the process of the MEC server using the rendering model to determine the video transmission time between the UAV and the user's VR device includes: (1.1) Determine the latency for the user's VR device to obtain the original video from the drone based on the data size of the VR content requested by the user, the video transmission rate, and the rendering decision.

[0039] In the local rendering mode, the drone needs to determine the rendering decision and transmit part of the original video task to the user. Let represent the rendering decision, and let represent the proportion of the video task that the drone needs to process for the user's VR device. The user's VR device needs to obtain the original content from the drone. At this time, the latency can be expressed as Equation (6), where is the data size of the VR content requested by the user's VR device. is the data size of the VR content requested by the user's VR device. is the data size of the VR content requested by the user's VR device.

[0040] (6) (1.2) Determine the presentation latency of the user's VR device based on the data size of the VR content requested by the user, the rendering resources of the user's VR device, the cycle for processing one bit of data, and the rendering decision.

[0041] After receiving the task from the drone, the user's VR device can use the local computing resources to complete the task processing. Let be the rendering resources of the user's VR device. Therefore, the presentation latency at the user's VR device can be expressed as Equation (7), where represents the number of CPU (Central Processing Unit / Processor) cycles required to process one bit of data.

[0042] (7) (1.3) Determine the video transmission time between the drone and the user's VR device based on the latency for the user's VR device to obtain the original video from the drone and the presentation latency of the user's VR device.

[0043] (2) In the MEC rendering mode, the process by which the MEC server uses the rendering model to determine the video transmission time between the drone and the user's VR device includes: (2.1) Determine the rendering latency of the MEC server based on the data size of the VR content requested by the user, the rendering resources of the user's VR device, the cycle for processing one bit of data, and the rendering decision. Among them, the rendering latency of the MEC server can be expressed as : (8) Among them, the results of the presented content are usually very large, and the transmission delay cannot be ignored.

[0044] (2.2) Determine the time when the drone transmits the rendered video to the user's VR device based on the data size of the VR content requested by the user, the video transmission rate, the data conversion rate, and the rendering decision.

[0045] When the drone completes the rendering process, it needs to transfer the result to the ground user. Accordingly, the time to transfer the rendered data from the drone to the user's VR device can be expressed by formula (9), where is the data conversion rate. is the data conversion rate.

[0046] (9) (2.3) Determine the video transmission time between the drone and the user's VR device based on the rendering delay of the MEC server and the time when the drone transmits the rendered video to the user's VR device.

[0047] In an exemplary embodiment of the present application, as Figure 2 shown, when transmitting the original task, a transmission channel may be occupied, so the drone must wait for the channel to be released before transmitting the presented result (such as Figure 2 in case 2). The drone needs to maintain a certain rate and meet the following conditions to ensure the transmission efficiency. For example, (such as Figure 2 in case 1). Then, the MTP delay of the user's VR device can be shown as formula (10). can be shown as formula (10).

[0048] (10) In an exemplary embodiment of the present application, a blockchain is introduced into the system to ensure a secure presentation process. To effectively improve resource efficiency, the proposed system deploys an effective Delegated Proof-of-Stake (DPoS) consensus mechanism. Based on this, a blockchain model can be constructed. Considering the limited resource capacity of drones, traditional consensus mechanisms such as Proof of Work (PoW) may not be directly applicable to drone networks. Based on this, in the blockchain, an effective Delegated Proof-of-Stake (DPoS) consensus mechanism is adopted to complete data sharing. In the system provided by the present application, an optimization algorithm based on block coordinate descent is used to select block producers. Block producers need to collect transactions and package them into a block. Then, the block will be shared with other validators to reach a consensus. The implementation process of this includes: (1) Block generation: Let and be the computing frequency of the block producer drone and the number of CPU cycles for processing one bit of block data respectively during the block generation process. Therefore, the delay in generating a block at the drone can be expressed by formula (11), where represents the size of the block.

[0049] (11) In the network, key information such as user requests, user location information, and drone decision information needs to be protected by the blockchain. Therefore, the size of the block can be expressed as formula (12), where represents the size of transaction processing.

[0050] (12) (2) Block verification: Block producers need to distribute the block to other validators in the network for the consensus process. Let be the transmission rate from drone to drone . The channel gain between drone and drone can be regarded as a free space path loss model. Therefore, the transmission rate can be defined as formula (13), where is the bandwidth between drone and drone .

[0051] (13) Then, the block transfer delay can be obtained, as expressed by Equation (14).

[0052] (14) The delay of block verification can be expressed by Equation (15), where is the block verification resource of the UAV, representing the total CPU cycles required during block verification.

[0053] (15) Let be the blockchain delay, indicating that all transactions are recorded in the blockchain. It can be expressed as Equation (16), where is an index variable. If , then the UAV is selected as the current block producer; otherwise .

[0054] (16) The blockchain latency is the time taken to protect the data. The shorter the latency, the higher its security.

[0055] In an exemplary embodiment of the present application, in the system provided by the present application, the rendering process requires a large amount of computing resources to ensure complex matrix calculations. At the same time, the blockchain also requires computing resources to complete the block generation and verification processes. Based on this, in order to achieve the best balance between the MTP latency and energy consumption, an energy consumption model can be implanted in the MEC server. The energy consumption model is used to determine the energy consumption during the entire rendering process. Among them, the energy consumption during the UAV rendering process is expressed as Equation (17), where represents the effective capacitance coefficient of the UAV .

[0056] (17) The energy consumption during the block generation and verification processes is expressed as : (18) In addition, the energy consumption of the user's VR device during the local rendering process can be expressed as : (19) Therefore, in the system provided by the present application, the energy consumption during the entire rendering process It can be represented by formula (20), where is the energy consumption of the drone during hover time.

[0057] (20) In an exemplary embodiment of the present application, in order to obtain the best trade-off performance between energy consumption and MTP delay, the rendering decision, resource allocation, and drone layout are jointly optimized. Among them, two typical metrics, namely MTP delay and energy consumption, are selected and modeled as a joint optimization problem (i.e., the problem model).

[0058] Moreover, an effective algorithm is developed to handle this non-convex problem. First, the tightly coupled variables are decoupled and transformed into two optimization problems. Then, the Block coordinate descent (BCD) optimization algorithm is used to solve this problem, which combines the Lagrangian multiplier iterative (LMI) method and the Deep reinforcement learning (DRL) method. By integrating the LMI method into the learning environment, the DRL method serves as the main routine, while the LMI method serves as the subroutine, which can effectively improve the efficiency of problem-solving. And, compared with other baseline solutions and traditional DRL methods [e.g., Proximal policy optimization (PPO) and Deep Q-Network (DQN)], the method proposed in this embodiment can achieve good performance for the drone-supported virtual reality system.

[0059] Based on the above description, within a series of constraints, an optimal trade-off between the MTP delay of the user's VR device and the system energy consumption is achieved. Let and represent the weight factors, which are designed to combine the optimization objectives. Then, the objective function of the problem model is represented by formula (21), where represents the mapping factor.

[0060]

[0061] In the formula, represents the optimization objective value, which is the weighted sum of the delay and energy consumption. represents the total number of users.

[0062] Let , , , , They are respectively for UAV deployment, rendering decision-making, computing resource allocation, bandwidth resource allocation, and block generation decision-making.

[0063] Based on the above description, the constructed problem model is shown in Equation (22). Among them, Equation (22a) is the rendering decision constraint. Equation (22b) means that there can only be one block producer in the network. This constraint condition in Equation (22c) ensures that the allocated computing resources do not exceed the maximum computing capacity of the UAV. Equation (22d) ensures that the transmission resources allocated for all user VR devices should not exceed the maximum limit. Equation (22e) ensures that the MTP delay cannot be greater than the maximum delay tolerance of the user VR device. Equation (22f) ensures the safe deployment of the UAV in the network. Equation (22g) guarantees the delay performance of the blockchain. Equation (22h) can guarantee the transmission efficiency between the UAV and the user.

[0064] (22) (22a) (22b) (22c) (22d) (22e) (22f) (22g) (22h) In the formula, represents the allocated computing resources, represents the total sum of computing resources, represents the total sum of bandwidth resources, represents the maximum tolerable time delay, represents the UAV 's horizontal coordinate, represents the safe distance between two adjacent UAVs, represents the secure guaranteed time delay of the blockchain system.

[0065] Based on the above description, the problem is a complex non-convex problem and is a problem solved by traditional algorithms. Although the DRL method is an effective method for solving non-convex problems, due to the existence of multi-variable problems, its convergence performance may be reduced. Based on this, to solve the problem , during the process of the MEC server generating and transmitting the rendered video using the transmission model, rendering model, and problem model, the problem Decoupled into the Unmanned Aerial Vehicle's Development, Rendering Decision and Block Producer Selection (URBS) problem (i.e., sub-problem ) and the Computing and Bandwidth Resource Allocation (CBRA) problem (i.e., sub-problem ).

[0066] Then, design an algorithm based on Lagrangian multiplier iteration (LMI) to solve the CBRA problem. For the URBS problem, this application designs an algorithm based on DRL. The proposed DRL-based algorithm and the (LMI)-based algorithm are executed in the form of BCD. As Figure 3 shown, the LMI-based algorithm can be implanted into the DRL environment to improve the solution efficiency. The DRL algorithm outputs decisions through a neural network and can obtain the optimal CBRA policy from the environment. Once the DRL training converges, the combined policy can be obtained.

[0067] In the actual application process, the solution processes of the above two sub-problems can be described as follows: A. The solution process of the URBS problem (i.e., sub-problem ): To solve the URBS problem, the Proximal Policy Optimization (PPO) algorithm can be adopted. The PPO algorithm is a popular DRL method based on policy gradients. According to formula (22), the URBS problem is formulated as: (23) (23a) (23b) (23c) (23d) (23e) (23f) Then, use the Markov decision process (MDP) to model the decision process of the URBS problem. Let be a triple, where represents the system state space, Represents the system action space, Represents the system reward. In the system provided in this application, the agent (i.e., the drone) learns actions from the environment (i.e., drone development, rendering decision-making, and preventing producers from making choices), aiming to maximize the discounted cumulative reward. According to the formulated sub-problems

i.e., formula (23)

[0068] The action space in the system consists of three elements: drone positioning, block producer decision-making, and rendering decision-making. That is, the position of the drone, block producer decision-making, and rendering decision-making. Therefore, the action space can be expressed as: (25) (2) Reward function: The goal of this application is to achieve the best trade-off between the MTP latency and energy consumption of the user's VR device. However, the optimization goal of DRL is to maximize the system reward, which is completely opposite to the goal of this application. Therefore, the system reward can be set as the reciprocal of the optimization goal. It should be noted that some constraints in the sub-problem may not be guaranteed, which means that the system performance (such as the security performance and MTP latency of the user's VR device) will be severely affected. The system reward with a penalty function can be expressed as: (26) Among them, is the reward coefficient.

[0069] (3) PPO-based algorithm: PPO is an actor-critic-based algorithm. The performer network needs to select system actions (i.e., drone position, block producer decision-making, and rendering decision-making) according to the current state generated by the learning environment (i.e., the distance between the user's VR device and the drone, blockchain latency, and the data size allocated to the drone). For the critic network, it needs to evaluate the quality of the actions.

[0070] As Figure 4 shown, the PPO algorithm contains three neural networks, namely a new performer network, a critic network, and an old performer network. Let 、 and are the new performer network, the critic network, and the old performer network respectively. The parameters of these three networks can be represented by respectively. According to the system state, the old and new performer networks can generate action policies represented by , where represents the transition probability. The old performer network can be regarded as a special regulator to limit the variability of the new performer network. To evaluate the performance of the new performer network, it is necessary to evaluate the performance of the critic network.

[0071] In the PPO algorithm, the agent interacts with the environment based on the actor network (i.e., the new performer network, the critic network, and the old performer network) to complete data collection. The data collected by the actor network can be used to update the parameters. Subsequently, the old parameters are updated according to the new parameters . Let , be the system state, the system action, and the immediate reward at time step t. During the training process, the new actor network needs the state information at time step . Then, a probability distribution based on possible actions is formed. The agent can select actions according to the generated distribution.

[0072] For network updates, the old parameters given by are updated using the method of gradient ascent, where represents the learning rate of the new performer network, and represents the gradient operator of the new performer network. To avoid performance collapse, a clipped surrogate loss function is introduced in the PPO algorithm. The clipped surrogate loss function is shown as : (27) In the formula, is the advantage function, and is the probability ratio. is a function, is the new performer network, and is the old performer network.

[0073] In formula (27), the advantage function can effectively ensure the stability of the PPO algorithm, that is: (28) In the formula, represents the reward discount factor, is the GAE parameter used to reduce variance during training, and is at The time error at a moment. In the advantage function, at the time error at a moment is expressed as: (29) where, is the state value of the system state determined by the critic network , is the state value of the system state determined by the critic network . represents the system reward at a moment.

[0074] For the function , it can be expressed by formula (30), where is set to 0.2. Note that the clipping function can improve the learning performance by restricting the probability ratio in a specific region.

[0075] (30) To explore better strategies, the PPO algorithm often adopts the method of policy entropy. Let be the entropy of the new actor network. The loss function of the updated old actor network can be expressed by formula (31). Where is the coefficient parameter.

[0076] (31) The critic network updates its parameters using the gradient descent method, and the way to update the function can be expressed as , where represents its new value, represents its old value, is the loss function of the critic network, is the learning rate, is the gradient operator of the critic network. In addition, the loss function can be expressed as: (32) In the formula, is the expectation.

[0077] B, CBRA) problem (i.e., sub-problem ) solving process: In the original problem, a series of variables are used to represent the upper limit value. Correspondingly, the following relationship can be obtained: (33) In the formula, is a variable, and max represents taking the maximum value.

[0078] Based on formula (33), the CBRA problem is reformulated as: (34) (34a) (34b) (34c) (34d) (34e) (34f) (34g) (34h) By analyzing the sub-problem , the following proposition can be obtained: Proposition 1: The CBRA problem can be regarded as a convex optimization problem. From Proposition 1, it can be seen that the sub-problem is a convex optimization problem. Then, an LMI-based algorithm is proposed to solve it. The Lagrangian function of the CBRA problem is : (35) where , , . , , and correspond to the Lagrangian variables of formula (34a), formula (34b), formula (34c), formula (34d), formula (34e), formula (34f), formula (34g) and formula (34h), respectively.

[0079] To obtain the rendering resource decision, taking the partial derivative of with respect to , the following relationship can be obtained: (36) In the formula, and are both Lagrangian multiplier variables.

[0080] Based on the Karush-Kuhn-Tucker (KKT) conditions, it can be obtained: (37) In the network, each UAV can only serve a limited number of ground VR users. Therefore, it can be assumed that each UAV has sufficient computing resources during the rendering process and can obtain . Furthermore, it can be concluded that . Based on the above analysis, the optimal rendering resources can be calculated as: (38) In the formula, represents the Lagrange multiplier variable.

[0081] Similarly, taking the partial derivative of in formula (35), we have: (39) Among them, is an intermediate variable (used to simplify the expression), , and are both Lagrange multipliers, is the transmission power, is the channel gain, is the noise density.

[0082] In addition, according to the KKT conditions, there are the following conditions: (40) Among them, can be expressed as: (41) Then, the Newton method can be used to find . Let and be the values at the -th iteration and the -th iteration respectively. The update process can be shown as: (42) In the formula, and are functions of respectively (only formula expressions, without meaning), and we have: (43) (44) To find , we also take the partial derivative of the Lagrange multiplier variable , and get: (45) In the formula, and are both Lagrange multiplier variables.

[0083] According to the KKT conditions, we have (46) From formula (46), it can be found that the Lagrange multiplier variables and cannot be zero at the same time, and further conclusions can be drawn as follows: (47) In the formula, is the optimal value of the intermediate variable.

[0084] For the blockchain system, taking the partial derivative with respect to the variable gives: (48) In the formula, and are both Lagrange multipliers, is the size of the block.

[0085] Then, based on the KKT conditions, the following conclusions can be drawn: (49) Therefore, the computing resources for block production can be expressed as : (50) Then, the subgradient method is used to update the Lagrange multiplier variables, and the update process can be expressed as: (51) In the formula, is the value of the Lagrange multiplier at the -th iteration, is the value of the Lagrange multiplier at the -th iteration, is the change amount between two iterations of the Lagrange multiplier , is the Lagrange multiplier taking a positive value, is the value of the Lagrange multiplier at the -th iteration, is the value of the Lagrange multiplier at the -th iteration, is the Lagrange multiplier The change amount between two iterations, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier The change amount between two iterations, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, is the Lagrange multiplier at the value of the iteration, represents the step size at the iteration. Among them: (52) In the formula, is the total sum of computing resources, is the total bandwidth resource, is the maximum tolerable delay, is the security guarantee delay of the blockchain system.

[0086] In an exemplary embodiment of the present application, the advantages of the solution provided by the present application are illustrated by simulation comparison with the prior art. Among them: (1) Comparative experiments and simulation results: (1.1) Proposed Scheme without Bandwidth Resource Optimization (PSwoBR): A unified allocation method for bandwidth resources is adopted, where each ground virtual reality user is allocated the same bandwidth resource.

[0087] (1.2) Proposed Scheme without Computing Resource Optimization (PSwoCR): A unified allocation method for computing resources is adopted, where each ground virtual reality user is allocated the same computing resource.

[0088] (1.3) Proposed Scheme without Rendering Decision Optimization (PSwoRD): Random rendering decision values are adopted on each drone.

[0089] (1.4) Proposed Scheme without Bandwidth (PSwoB): Without introducing blockchain, the key information in the network may be tampered with by malicious nodes.

[0090] Figure 5 In, the proposed scheme is compared with three different baseline schemes, PSwoBR, PSwoCR, and PSwoRD. It can be found that the proposed scheme of the present application obtains the best performance. In addition, it can be observed that compared with resource allocation and UAV position optimization, the rendering decision has a more sensitive impact on the system cost. Further, the impact of different bandwidth capacities on the system cost is also analyzed. Specifically, as the bandwidth resource increases, the system cost will also decrease. When the bandwidth resource is set too small in the present application (for example, set to 26 MHz), the performance difference between different schemes reaches the maximum value, and as the bandwidth resource increases, the performance difference between these schemes will narrow. This reflects that the proposed scheme of the present application has significant advantages in resource-constrained scenarios.

[0091] (2) Comparative Experiments and Simulation Results: (2.1) Proposed Scheme without UAV Location Optimization (PSwoUL): The location of the UAV is not optimized, and the geometric center method is adopted.

[0092] (2.2) Deep Q-Network (DQN)-Based Scheme: DQN estimates the state value (Q-value) by learning a neural network to select the optimal action.

[0093] (2.3) Proximal Policy Optimization (PPO)-Based Scheme: PPO is a policy gradient method for reinforcement learning that uses a clipped objective function to ensure stable and efficient learning.

[0094] Figure 6 In the abscissa is the number of iterations, and in the ordinate is the system return. The performance of the scheme proposed in this application is compared with that of four other schemes (i.e., DQN, PPO, PSwoRD, and PSwoUL schemes). Compared with traditional deep reinforcement learning schemes (such as DQN and PPO), the scheme proposed in this application achieves the best performance in terms of convergence speed and optimal value. The main reason is that the scheme proposed in this application combines the Lagrangian multiplier iteration algorithm, which reduces the decision space and improves the learning performance. In addition, compared with the PSwoRD and PSwoUL schemes, the scheme proposed in this application has a significant advantage in terms of the optimal value, although its convergence performance has not reached the best level. The main reason is that the decision space in the PSwoRD and PSwoUL schemes is smaller than that of the proposed scheme, so a better convergence speed is achieved. However, the PSwoRD scheme does not optimize the rendering decision, and the PSwoUL scheme does not optimize the location of the UAV. Both of these decisions are important factors affecting the system cost, and the impact of the rendering decision is more serious than that of the UAV location factor.

[0095] Based on the above description, in the security optimization framework of the UAV virtual reality system provided in this application, the security and privacy are enhanced by introducing blockchain. Then, on this basis, a joint optimization problem is proposed to obtain the optimal trade-off between energy consumption and rendering latency. During the optimization process, issues such as rendering decisions, resource allocation, and UAV placement are jointly considered. To solve this problem, an optimization algorithm based on BCD is also designed, in which the LMI method and the DRL method are combined to improve the solution efficiency. The simulation results show that the scheme provided in this application can achieve better performance compared with other baseline solutions (i.e., PSwoBR, PSwoCR, and PSwoRD schemes). In addition, compared with traditional DRL methods (such as DQN and PPO), the optimization algorithm provided in this application achieves optimal performance in terms of both convergence speed and optimal value.

[0096] Based on the same inventive concept, an embodiment of the present application also provides an implementation method of a drone virtual reality system implemented based on the above drone virtual reality system. The implementation solutions provided by this method for solving problems are similar to those recorded in the above system. Therefore, the specific limitations in one or more of the following embodiments of the implementation method of the drone virtual reality system can refer to the limitations on the drone virtual reality system in the foregoing text and will not be elaborated herein.

[0097] In an exemplary embodiment, an implementation method of a drone virtual reality system is provided, including: Step 1: Construct a transmission model, a rendering model, and a problem model. The problem model integrates optimization objectives. The optimization objectives include: drone deployment, rendering decision-making, computing resource allocation, bandwidth resource allocation, and block generation decision-making. The block generation decision-making is implemented in the blockchain. The transmission model is used to determine the video transmission rate between the drone and the user's VR device. The rendering model is used to determine the video transmission time between the drone and the user's VR device.

[0098] Step 2: Generate and transmit a rendered video using the transmission model, the rendering model, and the problem model.

[0099] In summary, in the drone virtual reality system and its implementation method provided by the present application, the blockchain is integrated into the system to provide distributed management and control functions. Key information during the virtual reality rendering process can be recorded in the blockchain in a timely manner to enhance security and privacy. To improve resource efficiency, the blockchain system adopts an efficient Delegated proof-of-stake (DPoS) consensus mechanism. In the proposed framework, how to ensure the performance of both the virtual reality and the blockchain system simultaneously is a major challenge. At the same time, energy consumption and Motion-to-photon (MTP) latency are typical performance metrics of the integrated framework. Therefore, a joint optimization problem is formulated to achieve the best compromise between energy consumption and MTP latency, where rendering decision-making, resource allocation, and drone placement are jointly optimized. Due to the tight coupling between variables, the optimization problem is non-convex and difficult to solve using traditional methods. For this reason, the present application decouples the formulated problem and designs an optimization algorithm based on Block coordinate descent (BCD). In this algorithm, the present application innovatively combines the Lagrangian multiplier iterative (LMI) method and the Deep reinforcement learning (DRL) method to improve the solution efficiency.

[0100] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structural diagram can be as shown in Figure 7 . The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store unmanned aerial vehicle virtual reality data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for implementing an unmanned aerial vehicle virtual reality system.

[0101] Those skilled in the art can understand that Figure 7 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0102] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0103] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0104] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0105] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0106] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0107] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0108] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A drone virtual reality system, characterized in that, The described unmanned aerial vehicle virtual reality system includes: an unmanned aerial vehicle, an MEC server, a blockchain, and a user VR device; The MEC server and the blockchain are both integrated in the unmanned aerial vehicle; the blockchain exchanges data with the MEC server; the unmanned aerial vehicle and the user VR device perform rendering video transmission; the user VR device is used to receive the rendering video; the blockchain is used to store the position information of the unmanned aerial vehicle and the resource allocation information of the unmanned aerial vehicle and the user VR device; A transmission model, a rendering model are implanted in the MEC server and a problem model is constructed; the MEC server is used to generate and transmit the rendering video by using the transmission model, the rendering model and the problem model; the transmission model is used to determine the video transmission rate between the unmanned aerial vehicle and the user VR device; the rendering model is used to determine the video transmission time between the unmanned aerial vehicle and the user VR device; the problem model integrates optimization objectives; the optimization objectives include: unmanned aerial vehicle deployment, rendering decision, computing resource allocation, bandwidth resource allocation, and block generation decision; the block generation decision is implemented in the blockchain.

2. The drone virtual reality system according to claim 1, characterized in that, The transmission link between the unmanned aerial vehicle and the user VR device adopts a probabilistic LoS channel model; The process of the MEC server using the transmission model to determine the video transmission time and video transmission rate between the unmanned aerial vehicle and the user VR device includes: Determine the distance between the unmanned aerial vehicle and the user VR device; Based on the distance, determine the channel coefficient of the transmission link between the unmanned aerial vehicle and the user VR device; Determine the probability of the LoS link between the unmanned aerial vehicle and the user VR device and the probability of the NLoS link between the unmanned aerial vehicle and the user VR device; Based on the channel coefficient, the probability of the LoS link, and the probability of the NLoS link, determine the channel power gain between the unmanned aerial vehicle and the user VR device; Based on the channel power gain, determine the downlink transmission rate of the unmanned aerial vehicle; the downlink transmission rate is used as the video transmission rate between the unmanned aerial vehicle and the user VR device.

3. The drone virtual reality system according to claim 1, wherein, The rendering model includes a local rendering mode and an MEC rendering mode.

4. The drone virtual reality system according to claim 3, characterized in that, In the local rendering mode, the process of the MEC server using the rendering model to determine the video transmission time between the unmanned aerial vehicle and the user VR device includes: Based on the data size of the VR content requested by the user, the video transmission rate, and the rendering decision, determine the delay for the user VR device to obtain the original video from the unmanned aerial vehicle; Based on the data size of the VR content requested by the user, the rendering resources of the user VR device, the period for processing one bit of data, and the rendering decision, determine the presentation delay of the user VR device; Based on the delay for the user VR device to obtain the original video from the unmanned aerial vehicle and the presentation delay of the user VR device, determine the video transmission time between the unmanned aerial vehicle and the user VR device.

5. The drone virtual reality system according to claim 3, wherein In the MEC rendering mode, the process by which the MEC server determines the video transmission time between the drone and the user's VR device using the rendering model includes: Determining the rendering latency of the MEC server based on the data size of the VR content requested by the user, the rendering resources of the user's VR device, the cycle for processing one bit of data, and the rendering decision; Determining the time for the drone to transmit the rendered video to the user's VR device based on the data size of the VR content requested by the user, the video transmission rate, the data conversion rate, and the rendering decision; Determining the video transmission time between the drone and the user's VR device based on the rendering latency of the MEC server and the time for the drone to transmit the rendered video to the user's VR device.

6. The drone virtual reality system according to claim 1, characterized in that, In the blockchain, an optimization algorithm based on block coordinate descent is used to determine the block producer; the block producer and the validator complete data sharing using the proof-of-stake delegation consensus mechanism to generate the block generation decision.

7. The drone virtual reality system according to claim 1, characterized in that, An energy consumption model is also implanted in the MEC server; the energy consumption model is used to determine the energy consumption during the entire rendering process.

8. The drone virtual reality system according to claim 1, wherein During the process in which the MEC server uses the transmission model, the rendering model, and the problem model to generate and transmit the rendered video, the problem model is decoupled into the development of the drone, the rendering decision, and the block producer selection problem, and the computing and bandwidth resource allocation problem; An algorithm based on Lagrangian multiplier iteration is used to solve the computing and bandwidth resource allocation problem, and an algorithm based on deep reinforcement learning is used to solve the development of the drone, the rendering decision, and the block producer selection problem to obtain a combined strategy.

9. The drone virtual reality system according to claim 8, characterized in that, The algorithm for deep reinforcement learning is the proximal policy optimization algorithm.

10. A method for implementing a virtual reality system of an unmanned aerial vehicle, characterized in that, The implementation method of the drone virtual reality system includes: Constructing a transmission model, a rendering model, and a problem model; the problem model integrates optimization objectives; the optimization objectives include: drone deployment, rendering decision, computing resource allocation, bandwidth resource allocation, and block generation decision; the block generation decision is implemented in the blockchain; the transmission model is used to determine the video transmission rate between the drone and the user's VR device; the rendering model is used to determine the video transmission time between the drone and the user's VR device; Generating and transmitting the rendered video using the transmission model, the rendering model, and the problem model.

Images