Unmanned aerial vehicle surface reconstruction method based on point cloud semantic communication
By constructing a point cloud semantic communication framework and a selective transmission module, combined with an end-to-end optimization strategy, the problems of incomplete information and insufficient flexibility in UAV 3D reconstruction are solved, achieving efficient and accurate surface reconstruction and adapting to diverse transmission needs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2025-12-26
- Publication Date
- 2026-05-01
AI Technical Summary
In 3D reconstruction tasks, UAVs face the problem of incomplete information collection leading to reconstruction failure. Furthermore, existing semantic communication algorithms lack flexibility and scalability in different application scenarios, making it difficult to meet diverse transmission needs.
A method for UAV surface reconstruction based on point cloud semantic communication is designed. A point cloud semantic encoder, decoder, bit rate estimator and category importance knowledge base are constructed. Combined with a selective transmission module and an end-to-end joint optimization strategy, efficient and accurate surface reconstruction is achieved.
In scenarios with limited transmission bandwidth, this technology optimizes data transmission efficiency, improves reconstruction accuracy and flexibility, adapts to various 3D reconstruction task scenarios and complex communication needs of different data types, and simplifies information transmission and reconstruction processes.
Smart Images
Figure CN121962430A_ABST
Abstract
Description
A UAV Surface Reconstruction Method Based on Point Cloud Semantic Communication Technical Field
[0001] This invention belongs to the field of network communication technology, specifically to the field of semantic communication technology, and in particular to a method for UAV surface reconstruction based on point cloud semantic communication. Background Technology
[0002] UAV surface reconstruction refers to the three-dimensional reconstruction of the surface of a target or scene based on data collected by a UAV. The UAV carries sensors (such as LiDAR, high-definition cameras, infrared cameras, etc.) to acquire two-dimensional or three-dimensional raw data (such as point clouds, image sequences, depth information) of the target (such as buildings, terrain, industrial equipment, disaster site ruins, etc.) or scene. Then, through technologies such as computer vision, point cloud processing, and deep learning, these discrete and incomplete raw data are transformed into a continuous, complete, and high-precision three-dimensional surface model of the target or scene.
[0003] Due to their wide aerial field of view, large coverage area, and high mobility, drones have been widely used for 3D reconstruction tasks in outdoor scenarios. Drones utilize onboard cameras, LiDAR, and other sensors to photograph the ground or objects, collecting multi-view data, which is then used to generate 3D models through computer vision and image processing techniques. Compared to traditional 3D reconstruction techniques, deep learning-based 3D reconstruction techniques typically offer higher reconstruction accuracy in complex scenes. To achieve high-precision 3D reconstruction, these techniques usually require powerful computing capabilities and abundant data input. However, considering the limited payload capacity and computing resources of drones, the large amount of collected information usually needs to be transmitted completely to a ground base station for processing, a process that places high demands on communication bandwidth. Existing 3D reconstruction research often focuses on reconstruction accuracy without considering the amount of data transmitted. However, in real-world scenarios, bandwidth is often limited, and incomplete data transmission can lead to incomplete or failed 3D reconstructions.
[0004] With the development of artificial intelligence technology and the increasing complexity of its applications, semantic communication, as a novel communication method, has been extensively studied. Semantic communication is a technology that optimizes data transmission efficiency by understanding and conveying the meaning and context of information. Compared to traditional communication, semantic communication can identify and eliminate redundant parts of information, transmitting only key information during data transmission, thereby significantly reducing transmission burden, saving bandwidth and energy, and achieving more efficient data transmission. Therefore, it is necessary to design a system framework for 3D reconstruction with minimal data transmission.
[0005] Most existing semantic communication algorithms are based on an end-to-end architecture, using deep neural network encoders and decoders as a classic method to compress input data into fixed-length semantic features. However, considering the limitations of UAVs' payload capacity and computational resources, and the varying data category requirements of 3D reconstruction tasks in different scenarios, the fixed-length features and transmission methods in existing semantic communication limit their flexibility and scalability in different application scenarios, making it difficult to meet diverse transmission needs and application scenarios. Therefore, it is crucial to selectively transmit data sources based on their categories.
[0006] In summary, UAVs face the following key problems in different 3D reconstruction task environments: (1) Incomplete UAV information collection will lead to 3D reconstruction failure, and most 3D reconstruction studies do not consider the issue of data transmission volume; (2) The strong coupling of semantics and semantic communication networks required for 3D reconstruction tasks makes it difficult to achieve flexible and scalable task scenario transformation and key data category transmission.
[0007] Therefore, there is an urgent need for a method for UAV surface reconstruction based on point cloud semantic communication, which can combine semantic communication with 3D reconstruction to simplify the process from information acquisition to task completion. Simultaneously, it should implement a scalable and flexible semantic communication algorithm to adapt to various 3D reconstruction task scenarios and the complex communication needs of different data types. Summary of the Invention
[0008] This invention addresses the problem of incomplete surface reconstruction caused by the limited flight time and computing resources of UAVs in scenarios with limited transmission bandwidth, and provides a method for constructing UAV surface reconstruction based on point cloud semantic communication.
[0009] Technical Solution: A method for UAV surface reconstruction based on point cloud semantic communication, comprising the following steps: (1) Based on the requirements of UAV surface reconstruction task scenario for data transmission volume and transmission efficiency, a point cloud-based semantic communication framework is constructed. The semantic communication framework includes a point cloud semantic encoder, a point cloud semantic decoder, a bit rate estimator, and a category importance knowledge base. The point cloud semantic encoder includes extracting local features and global feature vectors to obtain semantic features. The point cloud semantic decoder is composed of a multi-layer fully connected network, used to reconstruct the d-dimensional semantic features into the coordinates of K points and output point cloud patches. The category importance knowledge base stores the importance scores of each category to guide subsequent steps. Continue transmission; the bit rate estimator is used to estimate the entropy coding bit rate of the quantized features and generate a probability quality function (PMF) table for entropy coding; (2) Construct a selective transmission module. After the feature semantic encoding is completed, the selective transmission module selectively transmits the feature dimensions based on the preset category importance weights in the category importance knowledge base, so as to realize priority transmission of important categories and dimension reduction transmission of secondary categories; (3) Formulate an end-to-end joint optimization strategy for semantic communication and surface reconstruction, including the end-to-end training framework of the point cloud semantic encoder, point cloud semantic decoder and surface reconstruction decoder. The parameters of each module are optimized simultaneously through the joint loss function to achieve efficient and accurate UAV surface reconstruction.
[0010] Furthermore, the point cloud semantic encoder consists of a PointNet++ layer, a PointNet layer, and a quantization layer. The PointNet++ layer extracts small-scale local features and includes three core modules: a downsampling layer (FPS), a grouping layer (KNN), and a PointNet layer. The downsampling layer has K / 2 sampling points, the grouping layer has K / 4 ensemble points, and the PointNet layer consists of three multilayer perceptron (MLP) layers and a max-pooling layer. Finally, the PointNet layer extracts the global feature vector to obtain semantic features. Specifically, the semantic communication framework divides the input point cloud P into S patches and then extracts the low-dimensional semantic features of each patch. Where d is the feature dimension; the semantic features of all S patches constitute the feature matrix. The point cloud semantic decoder consists of a multi-layer fully connected network that reconstructs the coordinates of K points from the d-dimensional semantic features and outputs point cloud patches. The category importance knowledge base stores the importance scores of each category to guide subsequent transmission. The bit rate estimator estimates the entropy coding bit rate of the quantized features and generates a probability quality function (PMF) table for entropy coding.
[0011] For the semantic communication framework constructed above, the following process is executed: 1) First, sample the point cloud data to be sent from the point cloud dataset. N is the number of points in the point cloud, and 3 is the dimension. Point cloud After being transmitted to the point cloud semantic encoder, S center points are first selected using the FPS algorithm. Then, the KNN algorithm is used to construct K neighborhood points for each center point to form a patch. Finally, the patch coordinates are converted into local coordinates relative to the center points to obtain the patch set. .
[0012] 2) For each patch Extracting corresponding semantic features through a semantic encoder The semantic features of the S patches then form a semantic feature matrix. .
[0013] 3) A point cloud information selection and transmission mechanism based on category importance is used to selectively transmit the features encoded by each category of data by pre-defining importance weighting and reducing the dimensionality of the data for transmission, thereby improving the overall semantic transmission capability.
[0014] 4) The selected semantic features are transmitted to the quantization module for feature quantization.
[0015] During training, the quantization module employs uniform noise quantization, adding uniform random noise in the range of -0.5 to 0.5 to each feature value to simulate quantization error. During inference, the quantization module uses rounding quantization, performing a round operation on each feature value to obtain the quantized features. The quantized values range from the set of integers. Quantized features The data is transmitted to the bit rate estimator for entropy coding bit rate estimation. The bit rate estimator estimates the probability of occurrence of each quantized value by calculating the PMF function of the quantization features and generates a PMF table.
[0016] In actual transmission, after the bit rate estimator generates the PMF table, it converts the quantization features into a symbol sequence. Then, it encodes the symbol sequence and the PMF table into a compressed bitstream using arithmetic coding, achieving entropy coding. The sending end transmits the compressed bitstream and the center point coordinates of the patch. The sending and receiving ends share the PMF table. The receiving end decodes the bitstream into a symbol sequence using arithmetic decoding, and then recovers the quantization features, achieving entropy decoding.
[0017] The receiving end will decode the quantized features In the input point cloud semantic decoder, point cloud patches are reconstructed through a multi-layer fully connected network. Then, the patch coordinates are added to the center point coordinates of the patch to convert them from local coordinates to global coordinates, thus obtaining the reconstructed point cloud. .
[0018] On the other hand, it provides end-to-end joint optimization capabilities for UAV data acquisition and surface reconstruction, ensuring efficient semantic communication transmission while improving surface reconstruction quality and simplifying the process. The specific process can be described as follows: First, a point cloud semantic encoder is constructed. Point cloud semantic decoder and surface reconstruction decoder Point cloud semantic encoder Composed of the FPS algorithm, KNN algorithm, SAPP module, and PointNet module, it is used to extract semantic features of point clouds and transmit them. Surface reconstruction decoder It consists of a voxelization processing module, a 3D convolutional encoder, and a point cloud completion decoder, and is used to complete and reconstruct the output of the point cloud semantic encoder into a complete point cloud.
[0019] At the drone transmitter, a point cloud semantic encoder Extracting semantic features from point cloud P The semantic features are then quantified based on category importance. Then, a compressed bitstream is generated using entropy encoding and transmitted, along with the coordinates of the patch's center point. At the receiving end, the compressed bitstream is first decoded into quantized features using entropy decoding. Then, the point cloud is reconstructed using a point cloud semantic decoder. This yields a point cloud patch with local coordinates. The point cloud patch with local coordinates is then added to the center point coordinates of the received patch to obtain the reconstructed point cloud with global coordinates.
[0020] At the receiving end, the reconstructed point cloud Input Surface Reconstruction Decoder Perform surface reconstruction to obtain a complete point cloud. In semantic communication mode, the receiving end only uses the point cloud reconstructed by the point cloud semantic communication algorithm. As input to the surface reconstruction part, it achieves end-to-end semantic communication and surface reconstruction without relying on the original point cloud. This is achieved through a joint loss function. Simultaneously, the parameters of the point cloud semantic encoder, point cloud semantic decoder, and surface reconstruction decoder are optimized, among which... This is the bit rate loss, used to control compression efficiency; The loss for patch reconstruction in point cloud semantic communication is used to measure the quality of the point cloud semantic communication results. The loss in surface reconstruction is used to measure the quality of the final surface reconstruction, representing the complete point cloud reconstruction. Through end-to-end joint training, the semantic features extracted by the semantic communication encoder not only optimize compression efficiency but also improve the performance of surface reconstruction, achieving joint optimization of semantic communication and surface reconstruction.
[0021] Furthermore, the point cloud semantic communication module includes a semantic encoder, a channel, and a semantic decoder. The construction process is as follows: (11) S centroids are sampled from the original point cloud using the farthest point sampling algorithm to ensure that the centroids are evenly distributed. Then, K neighboring points are selected for each centroid using the nearest neighbor algorithm. The coordinates of the neighboring points are subtracted from the coordinates of the corresponding centroids to obtain S point cloud patches. Each patch contains K coordinate difference points and satisfies the following conditions: , Represents the total number of points in the original point cloud. Multiples of, and set To ensure consistency with the input point cloud resolution; (12) Input the point cloud patch into the semantic encoder to generate hidden layer features of d-dimensional local semantic vector combination. After uniform quantization of the hidden layer features, combine the centroid coordinates to form the potential features of the point cloud; (13) Perform entropy encoding and channel encoding on the potential features in sequence to generate a bit stream and transmit it to the receiving end through the channel; (14) The receiving end performs channel decoding and entropy decoding on the bit stream in sequence, separates the potential patch features and inputs them into the semantic decoder. Combine the centroid coordinates to decode and obtain the reconstructed point cloud patch. The union of all patches is the reconstructed point cloud output by the semantic communication module.
[0022] Furthermore, the encoding process of the semantic encoder is represented as follows:
[0023] In the formula, This indicates a point cloud patch. Represents a semantic coding network. For entropy coding networks, These are the training parameters for the entropy-encoding network. For channel coding networks, These are the training parameters of the channel coding network. The latent features corresponding to the bitstream generated after encoding; the decoding process of the semantic decoder is represented as follows:
[0024] In the formula, For channel decoder networks, These are the training parameters of the channel decoder network. This is an entropy decoding network. These are the training parameters of the entropy decoding network. It is a semantic decoding network. These are the training parameters of the semantic decoding network. It is the reconstructed point cloud obtained from decoding.
[0025] The process of extracting local and global features of the point cloud in step (1) is as follows: (11) Sample point clouds from the point cloud dataset. , It is the number of points in the point cloud; (12) from the point cloud by sampling from the farthest point FPS algorithm. Select S center points, and then use the K-Nearest Neighbors (KNN) algorithm to construct K neighboring points for each center point to form a patch. Convert the patch coordinates to local coordinates relative to the center points to obtain the patch set. (13) For each patch Local features are obtained by aggregating local features through the Spatial Attention Point Processing (SAPP) module. (14) Use the PointNet module to extract the concatenated global features to obtain the semantic features of each patch. The semantic features of the S patches then form a semantic feature matrix. .
[0026] Furthermore, the selective transmission mechanism is implemented as follows: (21) Obtain the importance weight of the category to which each point cloud patch belongs. All weights within a batch are normalized using the following formula:
[0027] In the formula, and These represent the minimum and maximum weights within the batch, respectively. It is a numerical stability constant; (22) the normalized weights Mapped to a preset transmission ratio range ,in It is a configurable base transmission ratio hyperparameter, and the formula for calculating the transmission ratio is: (23) Calculate the actual transmission characteristic dimension based on the transmission ratio:
[0028] in The feature dimension output by the encoder. Indicates rounding down. Ensure the number of transmission dimensions is within Within the range; (24) Generate a binary transmission mask. , in the mask One element is 1, and the rest are 0, which encodes the feature. Multiplying the transmission characteristics element-wise with the mask yields the transmission features. , This indicates element-wise multiplication, with untransmitted dimensions set to zero.
[0029] Furthermore, in the bit rate estimation stage, only the number of bits in the actual transmission dimension is calculated. The specific calculation process is as follows: For the quantized transmission characteristics... Calculate the probability mass function (PMF):
[0030] in, Indexed by feature dimensions, It is a bit estimation function; it calculates the actual number of bits transmitted for each point cloud patch:
[0031] Calculate the batch average number of bits per point (BPP):
[0032] in, For batch size, Points for each patch.
[0033] Furthermore, the joint loss function is:
[0034] in, , , These are the weighting coefficients for point cloud recovery loss, surface reconstruction loss, and bit rate loss in semantic communication, respectively. It is the loss in point cloud semantic recovery. It is the loss in reconstructing a complete point cloud from the surface. Bit rate loss.
[0035] Furthermore, point cloud recovery loss The formula used to constrain the geometric accuracy of point cloud recovery during semantic decoding is as follows:
[0036] In the formula, This represents the Chamfer distance calculation function. This represents the function for calculating Earth Mover's Distance. It is a point cloud recovered from semantic decoding. It is the original point cloud.
[0037] Furthermore, the loss in reconstructing a complete point cloud from the surface The formula used to constrain the quality of the complete point cloud output from surface reconstruction is:
[0038] In the formula, Represents a true and complete point cloud. This represents the complete point cloud of the reconstructed surface.
[0039] Furthermore, this method increases the initial training size during the end-to-end training process. The weight is reduced. The weights are adjusted to ensure the semantic communication module correctly recovers the point cloud semantic information; the weights are gradually increased during the later stages of training. The weight is adjusted to improve the overall surface reconstruction and completion quality of the system.
[0040] Furthermore, the method described in this invention can be applied to disaster assessment, urban modeling, or environmental monitoring scenarios, enabling priority transmission of point cloud features of damaged areas in disaster zones, important urban buildings, or infrastructure in environments with limited transmission bandwidth, thereby achieving high-quality terrain reconstruction or urban modeling.
[0041] The method provided by this invention combines point cloud semantic communication technology with an end-to-end training framework for surface reconstruction, and a semantic communication algorithm that selectively transmits data based on category importance permissions. This simplifies the information transmission and surface reconstruction process for UAVs, improves system efficiency and reconstruction accuracy, and can be better applied in scenarios with limited transmission bandwidth. This invention can be applied to post-disaster assessment, urban modeling, and environmental monitoring. In emergency situations following natural disasters, UAVs can be rapidly deployed to disaster areas for terrain reconstruction. Due to bandwidth limitations, traditional data transmission may result in information loss, while the selective transmission mechanism of this invention prioritizes the transmission of key features of damaged areas, thereby providing more accurate post-disaster assessment data and helping rescue agencies quickly formulate rescue plans. UAVs can prioritize the transmission of point cloud data of important buildings or infrastructure even with limited bandwidth, achieving high-quality urban modeling. Attached Figure Description
[0042] Figure 1 shows the end-to-end system architecture of semantic communication and 3D reconstruction; Figure 2 shows the reconstruction effect of the aircraft surface; Figure 3 shows the reconstruction effect of the ship surface. Detailed Implementation
[0043] The present invention will now be described in detail with reference to the accompanying drawings and specific examples.
[0044] This invention provides a method for constructing UAV surface reconstruction based on point cloud semantic communication, solving the problem that UAVs, due to limited flight time and computing resources, need to transmit complete information to successfully reconstruct surfaces in scenarios with limited transmission bandwidth. First, based on the data volume and transmission efficiency requirements of the task scenario, this invention designs a point cloud-based semantic communication framework to reduce data transmission volume while ensuring point cloud transmission quality. For the scenario requirements of surface reconstruction, a selective transmission module is designed. During training, training data is selectively transmitted based on a preset importance score for each category to improve the effectiveness and accuracy of semantic communication. Second, to simplify the UAV data acquisition and surface reconstruction process, a joint optimization strategy for semantic communication and surface reconstruction is designed to achieve more efficient and accurate UAV surface reconstruction.
[0045] Figure 1 is an end-to-end system architecture diagram of semantic communication and 3D reconstruction corresponding to the method described in this invention. Referring to Figure 1, the implementation of this system architecture includes the following steps: Step 1, constructing a semantic communication framework that includes a point cloud semantic encoder, a point cloud semantic decoder, a bit rate estimator, and a category importance knowledge base.
[0046] The point cloud semantic encoder includes modules such as the farthest point sampling (FPS) algorithm, the K-nearest neighbor (KNN) algorithm, the spatial attention point processing module SAPP, and PointNet. The semantic communication framework segments the input point cloud P into S patches and then extracts low-dimensional semantic features from each patch. Where d is the feature dimension; the semantic features of all S patches constitute the feature matrix. The point cloud semantic decoder consists of a multi-layer fully connected network that reconstructs the coordinates of K points from d-dimensional semantic features, outputting a point cloud patch. A category importance knowledge base stores the importance scores of each category to guide subsequent transmission; the bit rate estimator estimates the entropy-encoded bit rate of the quantized features and generates a probability quality function (PMF) table for entropy encoding.
[0047] The main framework of point cloud semantic communication algorithms consists of three parts: a semantic encoder (UAV side), a channel, and a semantic decoder (ground base station side). (Original point cloud) Includes three-dimensional coordinates, represented as N represents the number of points. At the encoding end, First, the patch extractor processes the data to generate S patches. And the corresponding centroid coordinates, each patch includes Each patch is then fed into the semantic encoder of the autoencoder, generating hidden layer features that are combinations of the d-dimensional local semantic vectors of each patch. These features are quantized by a uniform quantizer and combined with the coordinates of the sampled centroids to form the latent features of the entire point cloud. Subsequently, these latent features are entropy-coded and channel-coded to generate a bitstream, which is then transmitted to the decoder.
[0048] The point cloud semantic communication process can be represented by the following steps: First, sample S centroids from the original point cloud using the farthest point sampling algorithm to ensure uniform distribution. Then, select K nearest neighbor points for each centroid. Subtract the coordinates of the neighboring points from the coordinates of the corresponding centroid to obtain S point cloud patches. Each point cloud patch contains K coordinate difference points, and satisfies the following conditions: , Represents the total number of points in the original point cloud. Multiples of, and set To ensure consistency with the input point cloud resolution, the point cloud patch is input into the semantic encoder to generate hidden layer features composed of d-dimensional local semantic vectors. After uniform quantization of the hidden layer features, they are combined with the centroid coordinates to form latent point cloud features. The latent features are then entropy encoded and channel encoded sequentially to generate a bit stream, which is transmitted to the receiver via the channel. The receiver sequentially performs channel decoding and entropy decoding on the bit stream to separate the latent patch features and input them into the semantic decoder. Combined with the centroid coordinates, the reconstructed point cloud patch is obtained. The union of all patches is the reconstructed point cloud output by the semantic communication module.
[0049] Based on the above process, the entire transmission process can be represented as:
[0050] In the formula, This indicates a point cloud patch. Represents a semantic coding network. For entropy coding networks, These are the training parameters for the entropy-encoding network. For channel coding networks, These are the training parameters of the channel coding network. These are the potential features corresponding to the bitstream generated after encoding.
[0051] Entropy decoding and channel decoding are the inverse operations of entropy coding and channel coding. After acquiring the point cloud representation, latent patch features are extracted and input into the semantic decoder of the autoencoder. At the decoding end, each patch is decoded, and then combined with auxiliary coordinate information to obtain multiple independent results. The union of each independent prediction result is the final reconstructed point cloud result. .
[0052]
[0053] in This represents the channel decoder network, whose training parameters are: ; This represents the entropy decoding network, whose training parameters are... ; This represents a semantic decoder network, whose training parameters are: The recovered semantics As input for subsequent surface reconstruction.
[0054] The knowledge base serves as the foundation of semantic communication algorithms. The sending and receiving ends synchronously share and update the semantic knowledge base. In this system, the knowledge base is represented by the structure and parameters of a neural network.
[0055] In the region acquisition and block processing stage of point cloud data, the farthest point sampling (FPS) algorithm is used to obtain data from the original point cloud. Medium sampling The centroid And ensure that the centroids of the region are distributed as uniformly as possible. For each centroid point... Using the nearest neighbor KNN algorithm to find Neighboring points This forms the corresponding region. Then, the coordinates of the K points are subtracted from the coordinates of the sampling points. The coordinates are used to obtain a set of coordinate differences. This reduces the impact of patch offsets on the point cloud semantic encoder; these coordinate differences are the patches transmitted to the network model. After the above operations, the original point cloud is segmented into... There are 1 patch, and each patch has 1 patch. Each sampling point's coordinates are used as the center point to reconstruct the specific point location, thus enabling a better match between the output point cloud and the real point cloud. The centroid coordinates obtained from FPS sampling are used as auxiliary coordinate information for the patch. Simultaneously, during compression, by satisfying... The conditions are used to divide the patches, maximizing the coverage of the points and avoiding situations where some points cannot be captured. This ensures that the resolution is consistent with the input point cloud.
[0056] Step 2: Construct a selective transmission module. In point cloud semantic communication, a selective transmission mechanism based on category importance is implemented. After encoding, the transmission dimension of point cloud features of different categories is dynamically adjusted according to the preset category importance weights. This achieves an adaptive strategy of prioritizing the transmission of important categories and reducing the dimension of secondary categories, thereby improving the overall semantic transmission performance.
[0057] The core of the selective transfer mechanism is to selectively transfer feature dimensions based on the importance weights of the categories, after feature encoding and before quantization, while maintaining the integrity of the semantic encoder. Specifically, for each point cloud patch sample in the batch, the preset importance weights of its category are first obtained. ,in This represents the index of the category. To ensure comparability between weights of different categories, the weights within the batch are first normalized:
[0058] in and These represent the minimum and maximum weights within the batch, respectively. It is the numerical stability constant. The normalized weights. Mapped to a preset transmission ratio range ,in This is a configurable base transmission ratio hyperparameter, with a default value of 0.5. The formula for calculating the transmission ratio is:
[0059] Based on transmission ratio Calculate the number of feature dimensions actually transmitted for each sample:
[0060] in The feature dimension output by the encoder. Indicates rounding down. Ensure the number of transmission dimensions is within Within the range. For each sample, generate a binary transport mask. , among which the former One element is 1, and the rest are 0:
[0061] Encoded features before quantization Element-wise multiplication with the transport mask is performed to selectively preserve dimensions.
[0062]
[0063] in, This indicates element-wise multiplication, with untransmitted dimensions set to zero, resulting in no bit overhead during subsequent quantization, entropy encoding, and transmission. Quantized features It only contains the dimensional information actually transmitted, thus significantly reducing the number of bits transmitted.
[0064] In the bit rate estimation stage, the selective transmission mechanism further optimizes bit rate calculation. Traditional bit rate estimation is based on the complete feature dimension. Selective transmission, however, only calculates the number of bits in the actual transmission dimension. More specifically, for quantization features... The bit rate estimator calculates the probability quality function (PMF) for each dimension:
[0065] in Indicates the index of the feature dimension.
[0066] Since the untransmitted dimensions have been set to zero, their corresponding probability contributions are masked by the transmission mask. The formula for calculating the actual number of bits transmitted is:
[0067] The batch average number of bits per point (BPP) is:
[0068] in, For batch size, The number of points for each patch. Through selective transmission, high-weight important categories are transmitted almost all dimensions, ensuring reconstruction quality, while low-weight minor categories are transmitted only some dimensions, significantly reducing the number of transmitted bits.
[0069] Step 3: Construct an end-to-end training framework to provide end-to-end joint optimization capabilities for UAV data transmission and surface reconstruction. This ensures the efficiency of semantic communication transmission while improving the quality of surface reconstruction and simplifying the process, enabling integrated processing from partial point clouds to complete point clouds.
[0070] The system adopts a cascaded differentiable architecture to achieve joint optimization of compression efficiency and reconstruction quality. As shown in Figure 1, the overall system framework mainly consists of two parts: point cloud semantic communication and surface reconstruction. Point cloud semantic communication involves steps such as patch partitioning, semantic encoding, entropy encoding, channel encoding / decoding, entropy decoding, and semantic decoding to encode a portion of the point cloud into low-dimensional features and then decode it to obtain patches. The surface reconstruction part involves voxel encoding and decoding to complete the decoded patches into a complete point cloud.
[0071] In semantic communication scenarios, the semantic communication encoder encodes the geometric information of a point cloud into low-dimensional semantic features. After these features are transmitted through the channel, the receiving end can only obtain the semantically decoded point cloud and cannot access the original partial point cloud. Therefore, the quality of the point cloud output by semantic communication directly affects the subsequent surface reconstruction effect. In the design of the end-to-end joint loss function, the loss function should not only optimize the performance of each module but also promote cooperation between the two modules. Therefore, this invention designs three joint loss functions to constrain the overall performance of the system from different perspectives.
[0072]
[0073] in, , , These are the weighting coefficients for point cloud recovery loss, surface reconstruction loss, and bit rate loss in semantic communication, respectively.
[0074] This is the loss in point cloud semantic recovery, used to constrain the geometric accuracy of the decoded and recovered point cloud. It ensures that point cloud semantic communication accurately recovers local geometric details, providing reliable data input for subsequent surface reconstruction and completion. The point cloud recovered through semantic decoding is... The original point cloud is The loss value is calculated using Chamfer distance or Earth Mover's Distance.
[0075] This is the loss from reconstructing the complete point cloud from the surface, used to constrain the quality of the output complete point cloud and ensure that the system reconstructs a high-quality complete point cloud. The complete point cloud reconstructed from the surface is... The true and complete point cloud is The loss for surface reconstruction is also calculated using Chamfer distance or EMD. In end-to-end training, backpropagation simultaneously affects the parameters of both the point cloud semantic communication part and the surface reconstruction part, allowing the semantic communication compression strategy to proactively consider the completion requirements of surface reconstruction.
[0076] This is the bit rate loss, used to directly constrain compression efficiency. It is usually set to a small value to maintain a low bit rate while ensuring reconstruction quality.
[0077] Figures 2 and 3 show a comparison of the partial input point cloud, the point cloud reconstructed by the end-to-end system, and the ground truth value. The partial point cloud serves as the input to the point cloud semantic communication and surface reconstruction end-to-end system, the middle point cloud represents the reconstructed point cloud result, and the ground truth value is the original complete point cloud. Extensive validation and simulation results demonstrate that this framework can complete surface reconstruction with less point cloud data transmission, verifying its superior efficiency and performance, and expanding the system's application in different data types and scenarios.
Claims
1. A method for UAV surface reconstruction based on point cloud semantic communication, characterized in that, The following steps are included: (1) Based on the requirements of UAV surface reconstruction task scenario for data transmission volume and transmission efficiency, a point cloud-based semantic communication framework is constructed. The semantic communication framework includes a point cloud semantic encoder, a point cloud semantic decoder, a bit rate estimator, and a category importance knowledge base. The point cloud semantic encoder includes extracting local features and global feature vectors to obtain semantic features. The point cloud semantic decoder is composed of a multi-layer fully connected network, used to reconstruct the d-dimensional semantic features into the coordinates of K points and output point cloud patches. The category importance knowledge base stores the importance scores of each category to guide subsequent transmission. The bit rate estimator uses... (1) Estimate the entropy coding bit rate of the quantized features and generate a probability quality function (PMF) table for entropy coding; (2) Construct a selective transmission module. After the feature semantic encoding is completed, the selective transmission module selectively transmits the feature dimensions based on the preset category importance weights in the category importance knowledge base, so as to realize priority transmission of important categories and dimension reduction transmission of secondary categories; (3) Formulate an end-to-end joint optimization strategy for semantic communication and surface reconstruction, including the end-to-end training framework of the point cloud semantic encoder, point cloud semantic decoder and surface reconstruction decoder. The parameters of each module are optimized simultaneously through the joint loss function to achieve efficient and accurate UAV surface reconstruction.
2. The UAV surface reconstruction method based on point cloud semantic communication according to claim 1, characterized in that, The point cloud semantic communication module includes a semantic encoder, a channel, and a semantic decoder. The construction process is as follows: (11) S centroids are sampled from the original point cloud using the farthest point sampling algorithm to ensure that the centroids are evenly distributed. Then, K neighboring points are selected for each centroid using the nearest neighbor algorithm. The coordinates of the neighboring points are subtracted from the coordinates of the corresponding centroids to obtain S point cloud patches. Each point cloud patch contains K coordinate difference points and satisfies the following conditions: ， Represents the total number of points in the original point cloud. Multiples of, and set To ensure consistency with the input point cloud resolution; (12) Input the point cloud patch into the semantic encoder to generate hidden layer features of d-dimensional local semantic vector combination. After uniform quantization of the hidden layer features, combine the centroid coordinates to form the potential features of the point cloud; (13) Perform entropy encoding and channel encoding on the potential features in sequence to generate a bit stream and transmit it to the receiving end through the channel; (14) The receiving end performs channel decoding and entropy decoding on the bit stream in sequence, separates the potential patch features and inputs them into the semantic decoder. Combine the centroid coordinates to decode and obtain the reconstructed point cloud patch. The union of all patches is the reconstructed point cloud output by the semantic communication module.
3. The UAV surface reconstruction method based on point cloud semantic communication according to claim 2, characterized in that, The encoding process of the semantic encoder is represented as follows: In the formula, This indicates a point cloud patch. Represents a semantic coding network. For entropy coding networks, These are the training parameters for the entropy-encoding network. For channel coding networks, These are the training parameters of the channel coding network. The latent features corresponding to the bitstream generated after encoding; the decoding process of the semantic decoder is represented as follows: In the formula, For channel decoder networks, These are the training parameters of the channel decoder network. This is an entropy decoding network. These are the training parameters of the entropy decoding network. It is a semantic decoding network. These are the training parameters of the semantic decoding network. It is the reconstructed point cloud obtained from decoding.
4. The UAV surface reconstruction method based on point cloud semantic communication according to claim 1, characterized in that, The selective transmission mechanism is implemented as follows: (21) Obtain the importance weight of the category to which each point cloud patch belongs. All weights within a batch are normalized using the following formula: In the formula, and These represent the minimum and maximum weights within the batch, respectively. It is a numerical stability constant; (22) the normalized weights Mapped to a preset transmission ratio range ,in It is a configurable base transmission ratio hyperparameter, and the formula for calculating the transmission ratio is: (23) Calculate the actual transmission characteristic dimension based on the transmission ratio: in The feature dimension output by the encoder. Indicates rounding down. Ensure the number of transmission dimensions is within Within the range; (24) Generate a binary transmission mask. , in the mask One element is 1, and the rest are 0, which encodes the feature. Multiplying the transmission characteristics element-wise with the mask yields the transmission features. ， This indicates element-wise multiplication, where untransmitted dimensions are set to zero, achieving selective dimension retention.
5. The UAV surface reconstruction method based on point cloud semantic communication according to claim 4, characterized in that, In the bit rate estimation stage, only the number of bits in the actual transmission dimension is calculated. The specific calculation process is as follows: For the quantized transmission characteristics... Calculate the probability mass function (PMF): in, Indexed by feature dimensions, It is a bit estimation function; it calculates the actual number of bits transmitted for each point cloud patch: Calculate the batch average number of bits per point (BPP): in, For batch size, Points for each patch.
6. The UAV surface reconstruction method based on point cloud semantic communication according to claim 1, characterized in that, The joint loss function is: in, 、 、 These are the weighting coefficients for point cloud recovery loss, surface reconstruction loss, and bit rate loss in semantic communication, respectively. It is the loss in point cloud semantic recovery. It is the loss in reconstructing a complete point cloud from the surface. Bit rate loss.
7. The UAV surface reconstruction method based on point cloud semantic communication according to claim 6, characterized in that, Point cloud recovery loss The formula used to constrain the geometric accuracy of point cloud recovery during semantic decoding is as follows: In the formula, This represents the Chamfer distance calculation function. This represents the function for calculating Earth Mover's Distance. It is a point cloud recovered from semantic decoding. It is the original point cloud.
8. The UAV surface reconstruction method based on point cloud semantic communication according to claim 6, characterized in that, Loss in reconstructing a complete point cloud from a surface The formula used to constrain the quality of the complete point cloud output from surface reconstruction is: In the formula, Represents a true and complete point cloud. This represents the complete point cloud of the reconstructed surface.
9. The UAV surface reconstruction method based on point cloud semantic communication according to claim 1 or 6, characterized in that, During end-to-end training, the initial training phase increases... The weight is reduced. The weights are adjusted to ensure the semantic communication module correctly recovers the point cloud semantic information; the weights are gradually increased during the later stages of training. The weight is adjusted to improve the overall surface reconstruction and completion quality of the system.
10. The UAV surface reconstruction method based on point cloud semantic communication according to any one of claims 1-9, characterized in that, The method can be applied to disaster assessment, urban modeling, or environmental monitoring scenarios. It can prioritize the transmission of point cloud features of damaged areas in disaster zones, important urban buildings, or infrastructure in environments with limited transmission bandwidth, thereby achieving high-quality terrain reconstruction or urban modeling.