Unmanned aerial vehicle based cloud digital twin scene reconstruction training method

Abstract

This invention relates to the field of wireless communication technology and discloses a cloud-based digital twin reconstruction training method based on unmanned aerial vehicles (UAVs). 1) K UAVs use onboard LiDAR to scan the current environment and obtain a point cloud map of the current environment; after scanning, point cloud data is formed; 2) The base station obtains the current channel status, the computing resources of the cloud, the base station, and the UAVs, as well as the required scene reconstruction accuracy, and notifies the UAVs to prepare to unload data; 3) An optimization problem is constructed, and an appropriate algorithm is selected to solve it, obtaining the results of channel bandwidth, computing resources, and training data allocation; 4) The UAVs unload data to the cloud and the base station according to the solution results and perform training. After training, the UAVs and the base station upload the trained network parameters to the cloud, and the cloud aggregates the parameters to improve the point cloud reconstruction network.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and more specifically to a cloud-based digital twin scene reconstruction training method based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Digital twins are technologies that use digital models of physical objects, systems, or processes to reflect their state and behavior in real time through data. With the development of the Internet of Things, cloud computing, and big data technologies, digital twins are gradually becoming important tools in fields such as smart cities, intelligent manufacturing, and precision agriculture. The introduction of drones provides an efficient and flexible means for constructing digital twin scenarios.

[0003] Due to their high flexibility and maneuverability, drones can fly to most areas of land to perform tasks. The drone-assisted construction of digital twin scenarios mentioned in this invention is one such example. Using lidar mounted on the drone, it can collect point cloud data of the current area in real time and transmit this data wirelessly to a base station and a cloud server for cloud-based digital twin scenario reconstruction. After the digital twin scenario is reconstructed in the cloud, the powerful computing capabilities of the cloud server allow the cloud to predict and analyze the state of various entities within the current scenario, thereby providing decision-making support for various intelligent devices. For example, in a smart transportation scenario, the cloud uses digital twins to assist vehicles in lane changing and route planning.

[0004] Thanks to the development of edge intelligence technology and neural networks, other intelligent devices or facilities can assist in digital twin reconstruction in the cloud. The drone and base station proposed in this paper are among them. A deep point cloud compression framework (DPCC) is jointly deployed and trained on the drone, base station, and cloud. This framework, built on neural networks, can compress and reconstruct point cloud scenes. Training parameters from the drone and base station are transmitted to the cloud via wired or wireless channels. The cloud can then aggregate these parameters to form a point cloud reconstruction network. Once trained, this network can reconstruct the digital twin scene.

[0005] However, how to reasonably allocate the point cloud data scanned by drones to various devices for training remains an unsolved problem. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies and fill the gap in the field of training methods for drone-assisted cloud-based digital twin scene reconstruction, this invention considers factors such as channel resources, computing resources at each end, and the accuracy of the reconstructed scene, and provides an effective training data allocation method.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] A cloud-based digital twin scene reconstruction training method based on drones is proposed. This method is based on an interconnected system of cloud, base station and drones. The system includes K drones UAVS equipped with airborne LiDAR devices. DPCC is deployed on the drones, base stations and cloud. At the same time, the three ends have different computing capabilities. The drones and base stations are interconnected through a wireless channel, and the base stations and cloud servers are interconnected through a wired fiber optic channel.

[0009] The cloud-based digital twin scene reconstruction training method based on drones includes the following steps:

[0010] 1) K drones use onboard lidar to scan the current environment and obtain a point cloud map of the current environment. After the scan is completed, point cloud data is formed, which will be used as the training set for training the cloud-based point cloud reconstruction network.

[0011] 2) The base station obtains the current channel status, the computing resources of each end (cloud, base station and drone), and the required scene reconstruction accuracy, and notifies the drone to prepare to unload the data.

[0012] 3) Construct an optimization problem and select an appropriate algorithm to solve it, obtaining the results of channel bandwidth, computing resources and training data allocation.

[0013] 4) The UAV unloads the data to the other two ends (cloud and base station) based on the solution results and performs training. After training is completed, the UAV and the base station upload the trained network parameters to the cloud. The cloud aggregates the parameters to improve the point cloud and reconstruct the network.

[0014] Furthermore, the problem of unloading training data from drones to base stations and the cloud is modeled as follows:

[0015]

[0016] Under the following restrictions:

[0017]

[0018] F k (x k )≥β (3)

[0019] C k =N k x k (4)

[0020]

[0021]

[0022]

[0023] K drones participated in the training, and Ck This represents the total number of bits that each drone needs to unload, which is determined by the required reconstruction accuracy for the current scene, i.e., (3) and (4). In (3), x k This represents point cloud point compression using H bits, F k (x k () represents the relationship between reconstruction accuracy and the number of bits required to compress each point cloud point, and β represents the reconstruction accuracy required for the current scene. In summary, (3) represents the minimum threshold for scene reconstruction accuracy. (4) represents that each point cloud point scanned by the UAV is compressed using H bits, where N k This indicates the number of points in the point cloud. These represent the number of bits used for local training of the drone, the number of bits used for training offloaded from the drone to the base station, and the number of bits used for training offloaded from the drone to the cloud, respectively. Therefore, (2) represents the sum of the number of bits allocated to different ends for training by each drone, which is equal to the total number of bits that the drone originally needed to offload.

[0024] These represent the computing resources used by the base station to process the point cloud data unloaded from each drone, the computing resources used by the cloud to process the point cloud data unloaded from each drone, and the channel bandwidth allocated by the base station to each drone. r ,f c B v2r This represents the total computing resource limit of the base station, the total computing resource limit of the cloud, and the total bandwidth limit of the link. Therefore, (5), (6), and (7) indicate that the resources allocated to each UAV are limited.

[0025] Furthermore, α k D represents the weight of each drone. k This represents the time it takes for each drone to transmit and process data. (D) k Represented as:

[0026]

[0027] in, s k h represents the transmit power of each drone. k N represents the channel gain from the base station to the drone. k R represents the noise power from the base station to the drone. r2c This represents the wired fiber optic link speed from the base station to the cloud. l a r a c These represent the number of calculation cycles for each terminal.

[0028] Furthermore, the original optimization problem (1) is decomposed into two sub-problems. First, the training data allocation problem is solved, i.e. Then, based on the results of the training data allocation, the resource allocation problem is solved, i.e.

[0029] In step 3), the training data allocation is solved. The problem should be transformed into the following form for solution:

[0030]

[0031] Under the following restrictions:

[0032]

[0033]

[0034]

[0035]

[0036] In step 3), the channel bandwidth and computational resource allocation are calculated. The problem should be transformed into the following form for solution:

[0037]

[0038] Under the following restrictions:

[0039]

[0040]

[0041]

[0042]

[0043] Among them, variable t k ,m,w and g k n and q are both auxiliary variables.

[0044] The technical concept of this invention is as follows: Existing technologies still lack methods for training cloud-based digital twin scene reconstruction using drones. Therefore, this invention focuses on this aspect and provides a method for training cloud-based digital twin scene reconstruction using drones. This invention considers factors such as channel bandwidth, computing resources, and scene reconstruction accuracy, and reduces the training time of the cloud-based point cloud reconstruction network by rationally allocating training data to each end.

[0045] The beneficial effects of this invention are mainly reflected in the following aspects: This method fills the gap in the existing technology for training methods of UAV-assisted cloud-based digital twin scenarios. By jointly optimizing factors such as channel bandwidth and computing resources, training data is reasonably allocated to the three ends, thereby reducing the training time of the cloud-based point cloud reconstruction network.

[0046] This method rationally allocates point cloud data to three endpoints for training under limited resources. While considering the scene reconstruction accuracy, it optimizes channel bandwidth and computing resources to reduce the training time of the cloud-based digital twin reconstruction network. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the allocation of point cloud data for training according to the present invention;

[0048] Figure 2 This is a comparison of the algorithm used in the method of this invention with other algorithms. Detailed Implementation

[0049] The present invention will now be further described with reference to the accompanying drawings.

[0050] Reference Figures 1-2 A data transmission method for drone-assisted digital twin scenario construction is based on an existing wireless information transmission system. The data transmission system for drone-assisted digital twin scenario construction consists of K drones equipped with lidar, 1 base station, and 1 cloud server.

[0051] In this embodiment, K drones use onboard LiDAR to scan the scene information of the current flight area and generate point cloud data for training. After the drones complete their scanning, the base station, knowing the amount of data that the K drones need to unload, executes an algorithm. This algorithm aims to allocate training data under limited channel bandwidth and computing resources to reduce training time. The training time is calculated by the following formula:

[0052]

[0053] This formula shows that it is a piecewise function with a total of three intervals, D. k This depends on the largest of the three intervals. The first term represents the computational latency of the drone itself processing data. The second term represents the transmission latency of the base station processing data and the drone transmitting data to the base station, where the transmission latency depends on the time it takes for the last drone's data to arrive at the base station. The third term represents the latency of the cloud processing data, the transmission latency from the base station to the cloud, and the latency of the drone transmitting data to the cloud through the base station's wireless channel, which also depends on the drone with the longest transmission time.

[0054] In this embodiment, the optimization methods for training data, channel bandwidth, and computing resources are as follows:

[0055] The computational resources of K drones, base stations, and the cloud, as well as the wireless channel bandwidth optimization problem from K drones to the base station, are modeled as follows:

[0056]

[0057] Under the following restrictions:

[0058]

[0059] F k (x k )≥β (3)

[0060] C k =N k x k (4)

[0061]

[0062]

[0063]

[0064] The original optimization problem is decomposed into two sub-problems: first, optimizing the allocation of training data, and then optimizing the channel bandwidth and computational resources. A branch-and-bound algorithm is used for training data allocation, while a depth-first search algorithm is used for channel bandwidth and computational resource allocation.

[0065] The UAV-based cloud-based digital twin scene reconstruction training method in this embodiment fills a gap in existing technologies for UAV-assisted cloud-based digital twin scene reconstruction training. By optimizing channel bandwidth and computing resources, and rationally allocating training data to each terminal, the cloud-based scene training time is reduced.

[0066] In this embodiment, the number of test drones was set to 8, and the numerical convergence under different algorithms was compared.

[0067] Figure 1 This paper demonstrates the complete process of how the present invention distributes point cloud data to three endpoints for training. First, the airborne LiDAR scans and generates point cloud data. Then, the UAV distributes the point cloud data to each endpoint. After transmission, each endpoint begins training. During training, the network parameters θ on the UAV endpoint are... 3 and the network parameters θ at the base station 1 It will be uploaded to the cloud, and the cloud also has its own network parameters θ. 2 After receiving network parameters, the cloud aggregates them to improve the cloud DPCC network and then reconstruct the digital twin space.

[0068] Figure 2The convergence of the algorithm used in this invention is compared with other algorithms. It can be seen that the depth unfolding algorithm used in this invention has a significantly shorter training time to achieve final convergence than the simulated annealing algorithm, and its convergence speed is also superior to other algorithms. Therefore, this invention has certain advantages.

[0069] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered as limited to the specific forms stated in these embodiments. The scope of protection of this invention is also based on equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A cloud-based digital twin scene reconstruction training method based on unmanned aerial vehicles (UAVs), characterized in that, The system is based on the interconnection of cloud, base station and drone. The system includes K drones equipped with airborne LiDAR, one base station and one cloud server. The drone UAVS, base station and cloud server are all deployed with the deep point cloud compression framework DPCC. The method includes the following steps: 1) K drones use onboard LiDAR to scan the current environment and obtain a point cloud map of the current environment; after the scan is completed, point cloud data is generated, which will be used as the training set for training the cloud-based point cloud reconstruction network; 2) The base station obtains the current channel status, cloud computing resources, computing resources of the base station and drone, and the required scene reconstruction accuracy, and notifies the drone to prepare to unload data; 3) Construct an optimization problem and select an appropriate algorithm to solve it, obtaining the results of channel bandwidth, computing resources, and training data allocation; In step 3), the specific steps for modeling and solving the optimization problem are as follows: 3.1) Establish a mathematical model: ; Under the following restrictions: ; K drones participated in the training. This represents the total number of bits that each drone needs to unload, which is determined by the required reconstruction accuracy for the current scene, i.e., equations (3) and (4); Equation (3) represents the minimum threshold for scene reconstruction accuracy. In Equation (3), This represents using H bits for point cloud point compression. This represents the relationship between reconstruction accuracy and the number of bits required to compress each point cloud point. This represents the required reconstruction accuracy for the current scenario; Equation (4) indicates that each point cloud point scanned by the UAV is compressed using H bits, where Indicates the number of points in the point cloud; Equation (2) means that the sum of the number of bits allocated to different ends for training by each drone is equal to the sum of the number of bits that the drone originally needed to unload. , , These represent the computing resources used by the base station to process the point cloud data unloaded from each drone, the computing resources used by the cloud to process the point cloud data unloaded from each drone, and the channel bandwidth allocated by the base station to each drone. , , This represents the total computing resource limit of the base station, the total computing resource limit of the cloud, and the total bandwidth limit of the link; Representing the weight of each drone, This represents the time it takes for each drone to transmit and calculate the data. Represented as: ； in, , This indicates the transmit power of each drone. This represents the channel gain from the base station to the drone. This represents the noise power from the base station to the drone. This represents the wired fiber optic link speed from the base station to the cloud. , , These represent the number of calculation cycles for each end. These represent the number of bits used for local training of the drone, the number of bits used for training offloaded from the drone to the base station, and the number of bits used for training offloaded from the drone to the cloud, respectively. 3.2) Decompose the original optimization problem (1) into two sub-problems. First, optimize the allocation of training data. Further optimize channel bandwidth and computing resources. , , The specific solution method is as follows: The optimization problem (1) is transformed into the following form for solving the training data allocation problem. : Establish a mathematical model: ； Under the following restrictions: ； The optimization problem (1) is transformed into the following form, which is used to solve for the channel bandwidth and computational resource allocation. , , : Establish a mathematical model: ； Under the following restrictions: ； Among the variables and All are auxiliary variables; 3.3) After decomposing the original problem into two subproblems, Equation (9) and Equation (16), iterative solution is started until... Numerical convergence or reaching the maximum number of iterations; among which, the branch and bound algorithm is used for training data allocation, and the depth expansion algorithm is used for channel bandwidth and computing resource allocation; 4) The UAV unloads the data to the cloud and base station based on the solution results and performs training. After training is completed, the UAV and the base station upload the trained network parameters to the cloud. The cloud aggregates the parameters to improve the point cloud and reconstruct the network.

Images