Adaptive depth source channel joint coding method for three-dimensional point cloud wireless transmission

Abstract

The invention relates to a three-dimensional point cloud wireless transmission-oriented adaptive deep source channel joint coding method, which belongs to the field of source channel coding, and comprises the following steps: S1, constructing a deep source channel joint encoder model which comprises a point cloud down-sampling module and a feature extraction module; s2, obtaining a complex semantic vector # imgabs0 # of an input channel by using a power normalization method suitable for the disordered point cloud, and enabling the complex semantic vector # imgabs0 # to be transmitted on a noise channel; s3, performing initial estimation on the point cloud features and the point cloud coordinates; s4, designing a channel condition adaptive sensing mechanism; s5, training the deep source channel joint encoder model by using a training strategy supported by multiple compression ratios; and S6, constructing a deep source channel joint decoder model which comprises a feature extraction module, a feature extension module and a point cloud coordinate reconstruction module.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud semantic communication and relates to an adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission. Background Art

[0002] Semantic communication, or semantic layer transmission, is a communication paradigm that focuses on the information contained in the transmitted symbols. This communication paradigm can use semantic relevance to remove redundant data, and restore semantics through the background knowledge base at the receiving end to perform specific tasks. The amount of data of abstract semantic concepts to be transmitted is usually much smaller than the amount of precise feature data, so communication based on this paradigm can greatly improve transmission efficiency.

[0003] The latest progress in deep learning and its applications, such as natural language processing, speech recognition, and computer vision, has made it possible to realize multimodal semantic communication. At the same time, source-channel joint coding is considered to be a feasible scheme for semantic communication, which is used to capture and transmit semantic features. This scheme is highly robust to harsh channel environments, i.e., low signal-to-noise ratio areas, and can well solve the cliff effect.

[0004] In the process of wireless channel transmission, the transmission of semantic information faces variable channel conditions. Designing and training the source-channel joint coding model for specific channel conditions will lead to a large amount of storage space requirements. In addition, for point cloud wireless transmission for multi-user transmission scenarios, in order to obtain different compression rates, it is necessary to change specific network parameters and retrain the model, which will consume a lot of computing power.

[0005] The adaptive deep source-channel joint coding of 3D point cloud wireless transmission is an end-to-end data transmission method with the characteristics of joint source-channel coding. Compared with traditional point cloud wireless transmission, the adaptive deep source-channel joint coding scheme for 3D point cloud wireless transmission can meet the transmission requirements under different signal-to-noise ratio conditions, thereby achieving efficient semantic information transmission. In addition, in the multi-user transmission scenario, the scheme allows multiple compression rates to be adaptively adjusted under unified model parameters, thereby improving the efficiency and quality of data transmission. Summary of the invention

[0006] In view of this, an object of the present invention is to provide an adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] An adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission includes the following steps:

[0009] S1: Construct a deep source-channel joint encoder model, including a point cloud downsampling module and a feature extraction module;

[0010] S2: Obtain the complex semantic vector of the input channel using a power normalization method suitable for unordered point clouds Allow it to be transmitted over a noisy channel;

[0011] S3: Initial estimation of point cloud features and point cloud coordinates;

[0012] S4: Design a channel condition adaptive sensing mechanism;

[0013] S5: Train the deep source-channel joint encoder model using a training strategy supported by multiple compression rates;

[0014] S6: Construct a deep source-channel joint decoder model, including a feature extraction module, a feature expansion module and a point cloud coordinate reconstruction module.

[0015] Further, step S1 constructs a deep source-channel joint encoder model, specifically comprising the following steps:

[0016] S11: For a given reference point cloud Where N represents the total number of points in the point cloud, 3 represents the coordinate dimension of the point cloud, and the farthest point sampling strategy is adopted for P to obtain the downsampled point cloud Where r represents the downsampling rate;

[0017] S12: The downsampled point cloud P down The feature extraction module captures the semantic information and the spatial dynamic relationship of its geometric structure, denoted as C represents the dimension of point cloud features.

[0018] Further, the farthest point sampling strategy steps in step S11 are as follows:

[0019] S111: randomly select a point from the point cloud as the first sampling point P0, add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud;

[0020] S112: For each unsampled point, calculate the minimum distance from it to all current sampling points, and select the farthest point P1 as the next sampling point;

[0021] S113: For all unsampled points, update their minimum distance to the sampling point set;

[0022] S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

[0023] Further, the feature extraction module in step S12 is implemented by the following steps:

[0024] Randomly initialize a basic convolution kernel B = {B m |m=1,...,M}, where represents the weight matrix, M represents the number of weight matrices in the weight library B, C in With C out Represent the feature input and output dimensions respectively;

[0025] Position adaptive coefficient generation: Use K nearest neighbor algorithm to obtain point cloud P down The local neighborhood information is used to generate the position adaptation coefficient Score according to the position relationship between the center point and its neighboring points. ij =MLP(θ(p i ,p j )), where θ represents softmax normalization;

[0026] Dynamic convolution kernel generation: combined with weight matrix B m And its related position adaptation coefficient Score ij m Generate convolution weights

[0027] Use the Point Transformer module to further extract point cloud neighborhood features:

[0028]

[0029] Among them, f i Represented as the i-th encoded point cloud feature vector, x i Represented as the feature vector of the i-th input point, q i Denote the corresponding query vector, k j Represented as x i The jth key value vector of the nearest neighbor, v j is represented as the j-th value vector, θ is represented as SoftMax normalization, α: represents the MLPs layer, δ: represents the position encoding function, ⊙ and They represent element-by-element multiplication and addition respectively.

[0030] Further, in step S2, according to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are converted into Convert to hidden variables And Perform power normalization; the steps for implementing the power normalization method include:

[0031] S21: Record the sliding mean μ and variance σ of the point cloud during the training phase;

[0032] S22: Use (μ,σ) pairs in the inference phase Normalize to get z' and convert it into a complex-valued vector

[0033] The encoded symbol z is transmitted over a noisy channel and is represented by the function η: The i-th channel output received by the source-channel joint decoder is Where ω represents a differentiable noise function, which is used to implement back propagation in model training; h represents the channel gain;

[0034]

[0035] Further, step S3 specifically includes: outputting the channel and the signal-to-noise ratio condition μ as the input of the source-channel joint decoder. First, Convert to real vector Then perform deconvolution operation θ: Get the initial estimate y' of the point cloud feature, and finally pass y' through the coordinate reconstruction layer ψ: Get the initial estimate p' of the point cloud coordinates.

[0036] Further, step S4 specifically includes the following steps:

[0037] S41: Based on the point cloud coordinates and initial feature estimates, the self-attention mechanism is used to obtain new features of the input point cloud;

[0038] S42: Expand the signal-to-noise ratio condition μ to the same dimension as the point cloud feature and add it element by element;

[0039] S43: Calculate the scaling factor S according to the point cloud context information;

[0040] S44: Multiply the input point cloud feature by the scaling factor element by element to obtain the point cloud attention feature.

[0041] Further, step S41 specifically includes: applying a maximum pooling operation to p' to obtain a global feature, replicating the global feature rN times, and connecting it with the initial feature y' to obtain a new feature f global , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected to f through residual connections. global Add to get the output point features

[0042] Further, the training strategy supported by multiple compression rates in step S5 includes the following steps:

[0043] S51: Define an available bandwidth ratio range ρ l ∈(0,0.05), where ρ l = k / C in N, k represents the dimension of the channel input symbol, C in Represents the dimension of the original input point cloud, and N represents the total number of points in the point cloud;

[0044] S52: After the steps S51 to S52 of the channel condition adaptive sensing mechanism design, according to the receiving end bandwidth resource ρ l , perform a mask operation on the features and make an initial estimate of the masked features and their coordinate values;

[0045] S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set

[0046]

[0047] S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, the distortion loss calculation expression under the i-th compression rate is:

[0048]

[0049] S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function

[0050]

[0051] Among them, α and β are positive numbers to ensure that When it is large, assign a large weight Given the current bandwidth ratio ρ l ; Γ is the maximum weight;

[0052] That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, assign the larger Give the loss under the current compression rate; otherwise reduce the corresponding weight

[0053] Further, in step S6, the implementation steps of the deep source-channel joint decoder model are as follows:

[0054] S61: Apply the maximum pooling operation to the initial estimated value y' of the point cloud feature obtained after processing in step S3 and copy it rN times, and then connect it with the initial estimated value y' of the point cloud feature to obtain a new feature

[0055] S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features

[0056] S63: f attention With f global Add element by element to get the output point features Pass F' through the adaptive module to obtain the encoded features

[0057] S64: The feature vector F obtained by the adaptive module A Input feature expansion module to get expanded point features Where r' represents the upsampling rate; the feature expansion module first replicates the input features r' times, and uses the transposed convolution branch to expand the features, and learns new point features in a learnable way; finally, the point features from the two branches are connected and fed into a set of MLPs to generate extended features:

[0058] F up =MLP(Concate[Dup(F A ,r');Deconv(F A )])

[0059] S65: Using residual learning strategy to expand the feature F up Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud:

[0060] ΔP=MLP(F up )

[0061]

[0062] The beneficial effect of the present invention is that: the present invention designs an adaptive deep source-channel joint coding method for wireless transmission of three-dimensional point clouds. When the object is a large amount of disordered point cloud data, the method based on source-channel separation is used for encoding and wireless communication. Not only can the end-to-end optimization fail to achieve the optimal performance of the system, but it also faces the problem of cliff effect in the communication system. During the transmission of wireless channels, the transmission of semantic information faces changeable channel conditions. Due to the uneven distribution of point clouds, designing multiple networks for different signal-to-noise ratios will cause a significant increase in storage costs. In multi-user transmission scenarios, it is also necessary to consider the differences in bandwidth conditions of different receiving ends.

[0063] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0065] Figure 1 A system flow chart for the present invention;

[0066] Figure 2 Schematic diagram of the channel condition adaptive sensing mechanism in step S4 of the present invention;

[0067] Figure 3 Schematic diagram of the training strategy supporting multiple compression rates in step S5 of the present invention. DETAILED DESCRIPTION

[0068] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0069] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0070] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0071] like Figure 1 As shown, the present invention provides an adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission, the core of which lies in the design of a channel condition adaptive perception mechanism and a multi-compression rate support scheme for multi-user transmission scenarios, and mainly includes the following steps:

[0072] S1: Determine the deep source-channel joint encoder model, including the selection of the point cloud downsampling strategy and the design of the feature extraction module. Considering that the wireless transmission object is a large amount of disordered point cloud data, and the use of the source-channel separation-based method for encoding and wireless communication not only fails to achieve the optimal performance of the system through end-to-end optimization, but also faces the problem of cliff effect in the communication system. Therefore, the source-channel joint encoder model is adopted. The encoder model consists of a point cloud downsampling module using the farthest point sampling and a feature extraction module based on dynamic kernel convolution. The encoding end function is f θ : Where k represents the dimension of the channel input symbol. Specifically, the following steps are included:

[0073] S11: For a given reference point cloud Where N represents the total number of points in the point cloud, 3 represents the coordinate dimension of the point cloud, and the farthest point sampling strategy is adopted for P to obtain the downsampled point cloud Where r represents the downsampling rate; the farthest point sampling strategy steps are as follows:

[0074] S111: randomly select a point from the point cloud as the first sampling point P0, add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud;

[0075] S112: For each unsampled point, calculate the minimum distance from it to all current sampling points, and select the farthest point P1 as the next sampling point;

[0076] S113: For all unsampled points, update their minimum distance to the sampling point set;

[0077] S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

[0078] S12: The downsampled point cloud P down The feature extraction module is used to capture rich semantic information and spatial dynamic relationships in geometric structures, denoted as C represents the dimension of point cloud features. The feature extraction module is implemented as follows:

[0079] Randomly initialize a basic convolution kernel B = {B m |m=1,...,M}, where represents the weight matrix, M represents the number of weight matrices in the weight library B, C in With C out Represent the feature input and output dimensions respectively;

[0080] Position adaptive coefficient generation: Use K nearest neighbor algorithm to obtain point cloud P down The local neighborhood information is used to generate the position adaptation coefficient Score according to the position relationship between the center point and its neighboring points. ij =MLP(θ(p i ,p j )), where θ represents softmax normalization;

[0081] Dynamic convolution kernel generation: combined with weight matrix B m And its related position adaptation coefficient Score ij m Generate convolution weights

[0082] The adaptive convolution kernel enables the convolution operation on point cloud features to more effectively capture the spatial dynamic relationship on the point cloud geometric structure and capture rich semantic information.

[0083] Use the Point Transformer module to further extract point cloud neighborhood features:

[0084]

[0085] Among them, f i Represented as the i-th encoded point cloud feature vector, x i Represented as the feature vector of the i-th input point, q i Denote the corresponding query vector, k j Represented as x i The jth key value vector of the nearest neighbor, v jis represented as the j-th value vector, θ is represented as SoftMax normalization, α: represents the MLPs layer, δ: represents the position encoding function, ⊙ and They represent element-by-element multiplication and addition respectively.

[0086] Figure 1 In it, TConv 128×128 and Conv 1×1 represent the transposed convolution and convolution operations and their convolution kernel sizes respectively, C×128 and 128×3 represent the feature input and output dimensions, and S128 and S1 represent the convolution step size.

[0087] S2: Select a power normalization method suitable for the unordered point cloud to obtain the complex semantic vector of the input channel According to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are Convert to hidden variables And Normalize the power to satisfy the average power constraint Where ω represents a differentiable noise function, which is used to implement back propagation in model training; h represents the channel gain. Specifically, the steps to implement the power normalization method are:

[0088] During the training phase, the sliding mean μ and variance σ of the point cloud are recorded.

[0089] In the inference phase, the (μ,σ) pair is used Normalize to get z' and convert it into a complex-valued vector

[0090] The encoded symbol z is transmitted over a noisy channel and is represented by the function η: The i-th channel output received by the source-channel joint decoder is Where ω represents a differentiable noise function, which is used to implement backpropagation in model training; h represents the channel gain.

[0091]

[0092] S3: Initially estimate the point cloud features and point cloud coordinates. First, the channel output Convert to real vector Then perform deconvolution operation θ: Get the initial estimate y' of the point cloud feature, and finally pass y' through the coordinate reconstruction layer ψ: Get the initial estimate p' of the point cloud coordinates.

[0093] S4: Figure 2As shown, a channel condition adaptive perception mechanism is designed, and the channel condition adaptiveness is realized by using an element-by-element addition method. The specific implementation includes:

[0094] Using the element-by-element addition method, we first use the self-attention mechanism to obtain the new features f of the input point cloud according to the point cloud coordinates and the initial feature estimates. i ', and then expand μ to the point cloud feature f i 'Add the same dimension element by element to get a new feature vector It not only contains the original point cloud features, but also reflects the influence of signal-to-noise ratio conditions.

[0095]

[0096] Calculate the scaling factor S based on the context information:

[0097] S = sigmoid(s)

[0098] Multiply the input point cloud feature by the scaling factor element by element to get the point cloud attention feature F A :

[0099] F A =S i ·f i '

[0100] Optionally, in step S4, a new feature f of the input point cloud is obtained by using a self-attention mechanism according to the point cloud coordinates and the initial feature estimation value. i ', specifically including: applying the maximum pooling operation to p' to obtain the global feature, copying the global feature rN times, and connecting it with the initial feature y' to obtain the new feature f global , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected to f through residual connections. global Add to get the output point features

[0101] Figure 2 middle represents element-by-element addition, Conv 1×1 represents the convolution operation and its convolution kernel size, 128×128 represents the feature input and output dimensions, and S1 represents the convolution step size.

[0102] S5: Figure 3 As shown, a training strategy supporting multiple compression rates is designed. The following steps are included:

[0103] S51: Define an available bandwidth ratio range ρ l ∈(0,0.05), where ρ l = k / C inN, k represents the dimension of the channel input symbol, C in Represents the dimension of the original input point cloud, and N represents the total number of points in the point cloud.

[0104] S52: After the steps S1 to S2 of the channel condition adaptive sensing mechanism design, according to the receiving end bandwidth resource ρ l , perform a simple masking operation on the features and make an initial estimate of the masked features and their coordinate values.

[0105] S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set

[0106] S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, the distortion loss calculation expression under the i-th compression rate is:

[0107]

[0108] S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function

[0109]

[0110] Among them, α and β are positive numbers to ensure that When it is larger, assign a larger weight Given the current bandwidth ratio ρ l Γ is the maximum weight value, which is used to ensure stable training.

[0111] That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, a larger Give the loss under the current compression rate; on the contrary, it indicates that the current compression rate is more dominant and the corresponding weight needs to be reduced

[0112] S6: Determine the deep source channel joint decoder model, including the design of feature extraction module, feature expansion module and the selection of point cloud coordinate reconstruction strategy. The implementation steps are as follows:

[0113] S61: First, apply the maximum pooling operation to the initial estimated value y' of the point cloud feature obtained after processing in step S3 and copy it rN times, and then connect it with the initial estimated value y' of the point cloud feature to obtain a new feature

[0114] S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features

[0115] S63: f attention With f global Add element by element to get the output point features Pass F' through the adaptive module to obtain the encoded features

[0116] S64: The feature vector F obtained by the adaptive module A Input feature expansion module to get expanded point features Where r' represents the upsampling rate. The feature expansion module combines the advantages of replication-based and learning-based methods. First, the input features are replicated r' times. At the same time, the transposed convolution branch is used to expand the features and learn new point features in a learnable way. Finally, the point features from the two branches are concatenated and fed into a set of MLPs to generate the expanded features:

[0117]

[0118] S65: Using residual learning strategy to expand the feature F up Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud:

[0119] ΔP=MLP(F up )

[0120]

[0121] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0122] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0123] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0124] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0125] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0126] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0127] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0128] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0129] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0130] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0131] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. An adaptive depth source-channel joint coding method for wireless transmission of three-dimensional point clouds, characterized by: The following steps are involved: S1: Construct a deep source-channel joint encoder model, including a point cloud downsampling module and a feature extraction module; S2: Obtain the complex semantic vector of the input channel using a power normalization method suitable for unordered point clouds Allow it to be transmitted over a noisy channel; S3: Initial estimation of point cloud features and point cloud coordinates; S4: Design a channel condition adaptive sensing mechanism; S5: Train the deep source-channel joint encoder model using a training strategy supported by multiple compression rates; S6: Construct a deep source-channel joint decoder model, including a feature extraction module, a feature expansion module and a point cloud coordinate reconstruction module.

2. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S1 constructs a deep source-channel joint encoder model, specifically including the following steps: S11: For a given reference point cloud Where N represents the total number of points in the point cloud, 3 represents the coordinate dimension of the point cloud, and the farthest point sampling strategy is adopted for P to obtain the downsampled point cloud Where r represents the downsampling rate; S12: The downsampled point cloud P down The feature extraction module captures the semantic information and the spatial dynamic relationship of its geometric structure, denoted as C represents the dimension of point cloud features.

3. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 2 is characterized in that: The farthest point sampling strategy steps in step S11 are as follows: S111: randomly select a point from the point cloud as the first sampling point P0, add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud; S112: For each unsampled point, calculate the minimum distance from it to all current sampling points, and select the farthest point P1 as the next sampling point; S113: For all unsampled points, update their minimum distance to the sampling point set; S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

4. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 2, characterized in that: The feature extraction module in step S12 is implemented by the following steps: Randomly initialize a basic convolution kernel B = {B m |m=1,...,M}, where represents the weight matrix, M represents the number of weight matrices in the weight library B, C in With C out Represent the feature input and output dimensions respectively; Position adaptive coefficient generation: Use K nearest neighbor algorithm to obtain point cloud P down The local neighborhood information is used to generate the position adaptation coefficient Score according to the position relationship between the center point and its neighboring points. ij =MLP(θ(p i ,p j )), where θ represents softmax normalization; Dynamic convolution kernel generation: combined with weight matrix B m And its related position adaptation coefficient Score ij m Generate convolution weights Use the Point Transformer module to further extract point cloud neighborhood features: Among them, f i Represented as the i-th encoded point cloud feature vector, x i Represented as the feature vector of the i-th input point, q i Denote the corresponding query vector, k j Represented as x i The jth key value vector of the nearest neighbor, v j is represented as the j-th value vector, θ is represented as SoftMax normalization, represents the MLPs layer, represents the position encoding function, ⊙ and They represent element-by-element multiplication and addition respectively.

5. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: In step S2, according to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are converted into Convert to hidden variables And Perform power normalization; the steps for implementing the power normalization method include: S21: Record the sliding mean μ and variance σ of the point cloud during the training phase; S22: Use (μ,σ) pairs in the inference phase Normalize z' and convert it into a complex-valued vector The encoded symbol z is transmitted over a noisy channel, using the function The i-th channel output received by the source-channel joint decoder is Where ω represents a differentiable noise function, which is used to implement back propagation in model training; h represents the channel gain; 6. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S3 specifically includes: outputting the channel and the signal-to-noise ratio condition μ as the input of the source-channel joint decoder. First, Convert to real vector Then perform deconvolution operation Get the initial estimated value y' of the point cloud feature, and finally pass y' through the coordinate reconstruction layer Get the initial estimate p' of the point cloud coordinates.

7. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S4 specifically includes the following steps: S41: Based on the point cloud coordinates and initial feature estimates, the self-attention mechanism is used to obtain new features of the input point cloud; S42: Expand the signal-to-noise ratio condition μ to the same dimension as the point cloud feature and add it element by element; S43: Calculate the scaling factor S according to the point cloud context information; S44: Multiply the input point cloud feature by the scaling factor element by element to obtain the point cloud attention feature.

8. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S41 specifically includes: applying the maximum pooling operation to p' to obtain the global feature, copying the global feature rN times, and connecting it with the initial feature y' to obtain the new feature f global , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected to f through residual connections. global Add to get the output point features 9. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: The training strategy supported by multiple compression rates in step S5 includes the following steps: S51: Define an available bandwidth ratio range ρ l ∈(0,0.05), where ρ l = k / C in N, k represents the dimension of the channel input symbol, C in Represents the dimension of the original input point cloud, and N represents the total number of points in the point cloud; S52: After the steps S51 to S52 of the channel condition adaptive sensing mechanism design, according to the receiving end bandwidth resource ρ l , perform a mask operation on the features and make an initial estimate of the masked features and their coordinate values; S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, the distortion loss calculation expression under the i-th compression rate is: S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function Among them, α and β are positive numbers to ensure that When it is large, assign a large weight Given the current bandwidth ratio ρ l ; Γ is the maximum weight; That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, assign the larger Give the loss under the current compression rate; otherwise reduce the corresponding weight 10. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: In step S6, the implementation steps of the deep source-channel joint decoder model are as follows: S61: Apply the maximum pooling operation to the initial estimated value y' of the point cloud feature obtained after processing in step S3 and copy it rN times, and then connect it with the initial estimated value y' of the point cloud feature to obtain a new feature S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features S63: f attention With f global Add element by element to get the output point features Pass F' through the adaptive module to obtain the encoded features S64: The feature vector F obtained by the adaptive module A Input feature expansion module to get expanded point features Where r' represents the upsampling rate; the feature expansion module first replicates the input features r' times, and uses the transposed convolution branch to expand the features, and learns new point features in a learnable way; finally, the point features from the two branches are connected and fed into a set of MLPs to generate extended features: F up =MLP(Concate[Dup(F A ,r');Deconv(F A )]) S65: Using residual learning strategy to expand the feature F up Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud: ΔP=MLP(F up )

Images