Cloud digital twin scene reconstruction training method based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of wireless communication, and discloses a cloud digital twin reconstruction training method based on an unmanned aerial vehicle. The method comprises the following steps: 1) K unmanned aerial vehicles scan a current environment by using airborne laser radars to obtain a point cloud map in the current environment; after scanning is completed, point cloud data are formed; (2) the base station obtains the current channel state, the computing resources of the cloud end, the base station end and the unmanned aerial vehicle end and the required scene reconstruction precision and informs the unmanned aerial vehicle of preparing to unload data; 3) constructing an optimization problem, and selecting a corresponding algorithm for solving to obtain a channel bandwidth, computing resource and training data allocation result; and 4) the unmanned aerial vehicle unloads data to the cloud end and the base station end according to a solving result and carries out training, after training is completed, the unmanned aerial vehicle and the base station upload trained network parameters to the cloud end, and the cloud end aggregates the parameters to perfect the point cloud reconstruction network.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a cloud-based digital twin scene reconstruction training method based on an unmanned aerial vehicle. Background Art

[0002] Digital twin is a technology that uses digital models of physical objects, systems or processes to reflect their status and behavior in real time through data. With the development of the Internet of Things, cloud computing and big data technologies, digital twins have gradually become an important tool in the fields of smart cities, smart manufacturing and precision agriculture. The introduction of drones provides an efficient and flexible means for the construction of digital twin scenarios.

[0003] Due to the high flexibility and high maneuverability of drones, they can fly to most areas on land to perform tasks. The drone-assisted construction of digital twin scenes mentioned in the present invention is one of them. Relying on the laser radar installed on the drone, the drone can collect point cloud data of the current area in real time, and transmit the data to the base station and cloud server by wireless transmission to reconstruct the digital twin scene in the cloud. After the digital twin scene is reconstructed in the cloud, relying on the powerful computing power of the cloud server, the cloud can use digital twins to predict and analyze the status of each entity in the current scene, so as to make decisions for various smart devices. For example, in the smart transportation scenario, the cloud uses digital twins to assist vehicles in changing lanes, path planning, etc.

[0004] Thanks to the development of edge intelligence technology and neural networks, other intelligent devices or facilities can assist the cloud in digital twin reconstruction. The drone and base station proposed in this article are one of them. The deep point cloud compression framework (DPCC) is jointly deployed and trained on the drone, base station and cloud. This framework is built on neural networks and can compress and reconstruct point cloud scenes. The training parameters of the drone and base station are transmitted to the cloud through wired or wireless channels. The cloud can aggregate these parameters to form a point cloud reconstruction network. After the network training is completed, the digital twin scene can be reconstructed.

[0005] However, how to reasonably distribute the point cloud data scanned by drones to various terminals for training is still an unresolved problem. Summary of the invention

[0006] In order to overcome the shortcomings of the existing technology and fill the gap in the field of existing drone-assisted cloud-based digital twin scene reconstruction training methods, the present invention takes into account factors such as channel resources, computing resources at each end, and the accuracy of reconstructed scenes, and provides an effective training data allocation method.

[0007] The technical solution adopted by the present invention to solve the technical problem is:

[0008] The cloud-based digital twin scene reconstruction training method based on drones is based on an interconnected system of cloud, base station and drones. The system includes K drones UAVS equipped with airborne lidar equipment. DPCC is deployed on drones, base stations and cloud, and the three ends have different computing capabilities. The drones and base stations are interconnected by wireless channels, and the base stations and cloud servers are interconnected by wired optical fiber channels.

[0009] The UAV-based cloud digital twin scene reconstruction training method includes the following steps:

[0010] 1) K drones use airborne laser radar to scan the current environment and obtain a point cloud map of the current environment. After the scanning is completed, point cloud data is formed, which will be used as a training set for training the cloud-based point cloud reconstruction network.

[0011] 2) The base station obtains the current channel status, the computing resources of each end (cloud, base station and drone), and the accuracy of the required scene reconstruction and notifies the drone to prepare to unload data.

[0012] 3) Construct an optimization problem and select a suitable algorithm to solve it, and obtain the results of channel bandwidth, computing resources and training data allocation.

[0013] 4) The UAV unloads data to the other two ends (cloud and base station) based on the solution results and conducts training. After the training is completed, the UAV and the base station upload the trained network parameters to the cloud, and the cloud aggregates the parameters to improve the point cloud reconstruction network.

[0014] Furthermore, the problem of unloading training data from drones to base stations and the cloud is modeled as follows:

[0015]

[0016] Under the following restrictions:

[0017]

[0018] F k (x k )≥β (3)

[0019] C k =N k x k (4)

[0020]

[0021]

[0022]

[0023] Among them, there are K drones participating in the training, Ck represents the total number of bits that each UAV needs to offload, which is determined by the reconstruction accuracy required for the current scene, i.e., (3) and (4). In (3), x k Indicates the use of H bits for point cloud compression, F k (x k ) represents the relationship between the reconstruction accuracy and the number of bits required to compress each point cloud point, β represents the reconstruction accuracy required in the current scene, in summary, (3) represents the minimum threshold of scene reconstruction accuracy. (4) represents that each point cloud point scanned by the drone is compressed using H bits, where N k Indicates the number of points in the point cloud. They represent the number of bits for local training of the drone, the number of bits for training offloaded from the drone to the base station, and the number of bits for training offloaded from the drone to the cloud. Therefore, (2) means that the sum of the number of bits allocated to each drone for training at different terminals is equal to the total number of bits that the drone originally needs to offload.

[0024] They represent the computing resources of the base station for processing the point cloud data unloaded by each drone to the base station, the computing resources of the cloud for processing the point cloud data unloaded by each drone to the cloud, and the channel bandwidth allocated by the base station to each drone. r ,f c ,B v2r It represents the total computing resource limit of the base station, the total computing resource limit of the cloud, and the total bandwidth limit of the link. Therefore, (5), (6), and (7) represent that the resources allocated to each drone are limited.

[0025] Furthermore, α k Represents the weight of each drone, D k Represents the time it takes for each drone to transmit and process data. k It is expressed as:

[0026]

[0027] in, s k represents the transmission power of each UAV, h k Represents the channel gain from the base station to the drone. N k Represents the noise power from the base station to the drone. R r2c Represents the rate of the wired optical fiber link from the base station to the cloud. l , a r , a c Represents the number of calculation cycles of each end.

[0028] Furthermore, the original optimization problem (1) is decomposed into two sub-problems. First, the training data allocation problem is solved, namely Then, the resource allocation problem is solved according to the result of training data allocation, that is,

[0029] In step 3), solve the training data distribution The problem should be converted into the following form for solution:

[0030]

[0031] Under the following restrictions:

[0032]

[0033]

[0034]

[0035]

[0036] In step 3), solve the channel bandwidth and calculate resource allocation The problem should be converted into the following form for solution:

[0037]

[0038] Under the following restrictions:

[0039]

[0040]

[0041]

[0042]

[0043] Among them, the variable t k ,m,w and g k ,n,q are auxiliary variables.

[0044] The technical concept of the present invention is that the existing technology is still blank in the UAV-assisted cloud digital twin scene reconstruction training method, so the present invention focuses on this part and provides a UAV-assisted cloud digital twin scene reconstruction training method. The present invention takes into account relevant factors such as channel bandwidth, computing resources and scene reconstruction accuracy, and reasonably distributes training data to each end to reduce the cloud point cloud reconstruction network training time.

[0045] The beneficial effects of the present invention are mainly manifested in: this method fills the gap in the prior art in the UAV-assisted cloud-based digital twin scene reconstruction training method, and reasonably distributes training data to three ends by jointly optimizing channel bandwidth, computing resources and other factors, thereby reducing the training time of the cloud-based point cloud reconstruction network.

[0046] This method reasonably distributes point cloud data to three ends for training under limited resources. While considering the scene reconstruction accuracy, it optimizes channel bandwidth and computing resources to reduce the training time of the cloud-based digital twin reconstruction network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic diagram of allocating point cloud data for training in the present invention;

[0048] Figure 2 It is a comparison between the algorithm used in the method of the present invention and other algorithms. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings.

[0050] Reference Figure 1-2 ,A data transmission method for UAV-assisted digital twin scene construction is implemented based on the existing wireless information transmission system, that is, the data transmission system for UAV-assisted digital twin scene construction consists of K UAVs UAVS equipped with lidar, 1 base station and 1 cloud server.

[0051] In the method of this embodiment, K drones use airborne laser radar to scan the scene information of the current flight area and generate point cloud data for training. After the drone scans, the base station knows the amount of data that needs to be unloaded by the K drones and then executes the algorithm. The algorithm aims to allocate training data under limited channel bandwidth and computing resources to reduce training time. The training time is calculated by the following formula:

[0052]

[0053] From the formula, we can see that this is a piecewise function. The piecewise function has three intervals in total. k It depends on the largest part of the three intervals. Among them, the first item represents the calculation delay of the drone itself processing data, the second item represents the transmission delay of the base station processing data and the drone transmitting data to the base station, and the transmission delay depends on the time when the last drone data arrives at the base station. The third item represents the cloud processing data delay, the transmission delay from the base station to the cloud, and the delay of the drone transmitting data to the cloud through the base station wireless channel, which also depends on the drone with the longest transmission time.

[0054] In this embodiment, the optimization method of training data, channel bandwidth and computing resources is as follows:

[0055] The computing resources of K drones, base stations and the cloud and the optimization problem of the wireless channel bandwidth from K drones to the base station are modeled as follows:

[0056]

[0057] Under the following restrictions:

[0058]

[0059] F k (x k )≥β (3)

[0060] C k =N k x k (4)

[0061]

[0062]

[0063]

[0064] The original optimization problem is decomposed into two sub-problems: first, optimizing the training data allocation, and then optimizing the channel bandwidth and computing resources. The branch-and-bound algorithm is used for the training data allocation, and the deep expansion algorithm is used for the channel bandwidth and computing resource allocation.

[0065] The drone-based cloud digital twin scene reconstruction training method of this embodiment fills the gap in the existing technology in drone-assisted cloud digital twin scene reconstruction training. By optimizing channel bandwidth and computing resources, the training data is reasonably distributed to each terminal, reducing the cloud scene training time.

[0066] In this embodiment, the number of test drones is set to 8, and the numerical convergence under different algorithms is compared.

[0067] Figure 1 The complete process of how the present invention distributes point cloud data to three terminals for training is shown. First, the airborne laser radar scans and generates point cloud data, and then the drone distributes the point cloud data to each terminal. After the transmission is completed, each terminal starts training. During the training process, the network parameters θ of the drone terminal 3 and the network parameters θ at the base station 1 Will be uploaded to the cloud, which also has its own network parameters θ 2 ,After receiving the network parameters, the cloud aggregates them to improve the cloud DPCC network and ,reconstruct the digital twin space.

[0068] Figure 2The convergence comparison of the algorithm used in the method of the present invention compared with other algorithms is shown. It can be seen that the training completion time of the deep expansion algorithm used in the present invention is much shorter than that of the simulated annealing algorithm, and the convergence speed is also better than other algorithms, so the present invention has certain advantages.

[0069] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept and are for illustrative purposes only; the protection scope of the present invention should not be deemed to be limited to the specific forms described in this embodiment, and the protection scope of the present invention is also based on the equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A cloud-based digital twin scene reconstruction training method based on drones, characterized in that: Based on the interconnection system of cloud, base station and UAV, the system includes K UAVs equipped with airborne laser radar, a base station and a cloud server, where the UAVs, base station and cloud server all deploy the deep point cloud compression framework DPCC; The method comprises the following steps: 1) K drones use airborne laser radar to scan the current environment and obtain a point cloud map of the current environment. After the scanning is completed, point cloud data is formed, which will be used as a training set for the cloud-based point cloud reconstruction network; 2) The base station obtains the current channel status, computing resources of the cloud, base station and drone, and the accuracy of the required scene reconstruction and notifies the drone to prepare to unload data; 3) Construct an optimization problem and select the corresponding algorithm to solve it, and obtain the results of channel bandwidth, computing resources and training data allocation; 4) The drone unloads data to the cloud and base station based on the solution results and performs training. After the training is completed, the drone and base station upload the trained network parameters to the cloud, and the cloud aggregates the parameters to improve the point cloud and reconstruct the network.

2. The UAV-based cloud digital twin scene reconstruction training method as described in claim 1, characterized in that: In step 3), the specific steps of modeling and solving the optimization problem are as follows: 3.1) Establish mathematical model: Under the following restrictions: F k (x k )≥β (3) C k =N k x k (4) Among them, there are K drones participating in the training, C k represents the total number of bits that each UAV needs to unload, which is determined by the required reconstruction accuracy of the current scene, i.e., equations (3) and (4); Formula (3) represents the minimum threshold of scene reconstruction accuracy. In formula (3), x k Indicates the use of H bits for point cloud compression, F k (x k ) represents the relationship between the reconstruction accuracy and the number of bits required to compress each point cloud point, and β represents the reconstruction accuracy required in the current scene; Formula (4) indicates that each point cloud point scanned by the drone is compressed using H bits, where N k Indicates the number of point cloud points; Formula (2) means that the sum of the number of bits allocated to different terminals for training by each UAV is equal to the sum of the number of bits that the UAV originally needs to offload; They represent the computing resources of the base station for processing the point cloud data unloaded by each UAV to the base station, the computing resources of the cloud for processing the point cloud data unloaded by each UAV to the cloud, and the channel bandwidth allocated by the base station to each UAV; r ,f c ,B v2r It represents the total computing resource limit of the base station, the total computing resource limit of the cloud, and the total bandwidth limit of the link; α k Represents the weight of each drone, D k Represents the time it takes for each drone to transmit and calculate data; D k It is expressed as: in, s k represents the transmission power of each UAV, h k represents the channel gain from the base station to the drone, N k represents the noise power from the base station to the drone, R r2c represents the rate of the wired optical fiber link from the base station to the cloud, a l , a r , a c Represents the number of calculation cycles of each end, They represent the number of bits for local training of the drone, the number of bits for training offloaded from the drone to the base station, and the number of bits for training offloaded from the drone to the cloud. 3.2) Decompose the original optimization problem (1) into two sub-problems. First, optimize the training data distribution Re-optimize channel bandwidth and computing resources The specific solution method is as follows: The optimization problem (1) is transformed into the following form to solve the training data allocation Build a mathematical model: Under the following restrictions: The optimization problem (1) is transformed into the following form to solve the channel bandwidth and computing resource allocation: Build a mathematical model: Under the following restrictions: The variable t k ,m,w and g k ,n,q are auxiliary variables; 3.3) Decompose the original problem into two sub-problems (Eq. (9) and (16) and then start iterative solution until D k The numerical convergence or the maximum number of iterations is reached; wherein, the branch and bound algorithm is used for training data allocation, and the depth expansion algorithm is used for channel bandwidth and computing resource allocation.

Images