Adaptive depth source-channel joint coding method for 3D point cloud wireless transmission

Abstract

The present invention relates to an adaptive deep source channel joint coding method for wireless transmission of three-dimensional point clouds, which belongs to the field of source channel coding and comprises the following steps: S1: constructing a deep source channel joint encoder model, comprising a point cloud downsampling module and a feature extraction module; S2: obtaining a complex semantic vector of an input channel by using a power normalization method suitable for disordered point clouds so as to transmit it on a noisy channel; S3: performing an initial estimation of point cloud features and point cloud coordinates; S4: designing a perception mechanism that is adaptive to channel conditions; S5: training the deep source channel joint encoder model by using a training strategy supported by multiple compression rates; and S6: constructing a deep source channel joint decoder model, comprising a feature extraction module, a feature expansion module and a point cloud coordinate reconstruction module.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud semantic communication and relates to an adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission. Background Art

[0002] Semantic communication, or semantic-layer transmission, is a communication paradigm that focuses on the information contained within transmitted symbols. This communication paradigm leverages semantic relevance to remove redundant data and, at the receiving end, uses a background knowledge base to recover semantic meaning to perform specific tasks. The amount of data required to transmit abstract semantic concepts is typically much smaller than the amount of precise feature data, so communication based on this paradigm can significantly improve transmission efficiency.

[0003] Recent advances in deep learning and its applications, such as natural language processing, speech recognition, and computer vision, have made multimodal semantic communication possible. Joint source-channel coding is considered a viable approach for semantic communication, capturing and transmitting semantic features. This approach is robust to harsh channel environments, particularly those with low signal-to-noise ratios, and effectively mitigates the cliff effect.

[0004] During wireless channel transmission, the transmission of semantic information faces variable channel conditions. Designing and training a joint source-channel coding model for specific channel conditions requires a large amount of storage space. Furthermore, wireless transmission of point clouds for multi-user transmission requires varying compression rates by changing specific network parameters and retraining the model, which consumes a significant amount of computing power.

[0005] Adaptive deep source-channel joint coding for 3D point cloud wireless transmission is an end-to-end data transmission method that leverages the characteristics of source-channel coding. Compared to traditional point cloud wireless transmission, this adaptive deep source-channel joint coding scheme for 3D point cloud wireless transmission can adapt to transmission requirements under varying signal-to-noise ratios, thereby achieving efficient semantic information transmission. Furthermore, in multi-user transmission scenarios, this scheme allows for adaptive use of multiple compression rates within a unified model parameter set, improving data transmission efficiency and quality. Summary of the Invention

[0006] In view of this, an object of the present invention is to provide an adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] An adaptive depth source-channel joint coding method for wireless transmission of three-dimensional point clouds includes the following steps:

[0009] S1: Construct a deep source-channel joint encoder model, including a point cloud downsampling module and a feature extraction module;

[0010] S2: Obtain the complex semantic vector of the input channel using a power normalization method suitable for unordered point clouds Enable it to transmit over a noisy channel;

[0011] S3: Initial estimation of point cloud features and point cloud coordinates;

[0012] S4: Design a channel condition adaptive sensing mechanism;

[0013] S5: Train the deep source-channel joint encoder model using a training strategy that supports multiple compression rates.

[0014] S6: Construct a deep source-channel joint decoder model, including a feature extraction module, a feature expansion module, and a point cloud coordinate reconstruction module.

[0015] Furthermore, step S1 constructs a deep source-channel joint encoder model, specifically including the following steps:

[0016] S11: For a given reference point cloud Where N represents the total number of points in the point cloud, 3 represents the coordinate dimension of the point cloud, and the farthest point sampling strategy is adopted for P to obtain the downsampled point cloud Where r represents the downsampling rate;

[0017] S12: The downsampled point cloud P down The feature extraction module captures the semantic information and the spatial dynamic relationship of its geometric structure, denoted as C represents the dimension of point cloud features.

[0018] Furthermore, the farthest point sampling strategy steps in step S11 are as follows:

[0019] S111: Randomly select a point from the point cloud as the first sampling point P0, add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud;

[0020] S112: For each unsampled point, calculate its minimum distance to all current sampling points, and select the farthest point P1 as the next sampling point;

[0021] S113: For all unsampled points, update their minimum distance to the sampling point set;

[0022] S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

[0023] Furthermore, the feature extraction module in step S12 is implemented by the following steps:

[0024] Randomly initialize a basic convolution kernel B = {B m |m=1,...,M}, where Represents the weight matrix, M represents the number of weight matrices in the weight library B, C in with C out Represent the feature input and output dimensions respectively;

[0025] Position adaptive coefficient generation: Use K nearest neighbor algorithm to obtain point cloud P down The local neighborhood information is used to generate the position adaptive coefficient Score according to the position relationship between the center point and its neighboring points. ij =MLP(θ(p i ,p j )), where θ represents softmax normalization;

[0026] Dynamic convolution kernel generation: combined with weight matrix B m and its related position adaptation coefficient Score ij m Generate convolution weights

[0027] Use the Point Transformer module to further extract point cloud neighborhood features:

[0028]

[0029] Among them, f i Represented as the i-th encoded point cloud feature vector, x i Represented as the feature vector of the i-th input point, q i Denote the corresponding query vector, k j Represented as x i The jth key value vector of the nearest neighbor, v j is represented as the j-th value vector, θ is represented as SoftMax normalization, α: represents the MLPs layer, δ: represents the position encoding function, ⊙ and They represent element-wise multiplication and addition respectively.

[0030] Furthermore, in step S2, according to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are converted to Convert to latent variables And Perform power normalization; the power normalization method includes the following steps:

[0031] S21: records the sliding mean μ and variance σ of the point cloud during the training phase;

[0032] S22: Use (μ,σ) pairs in the inference phase Normalize to get z' and convert it into a complex vector

[0033] The coded symbol z is transmitted over a noisy channel and is represented by the function η: The i-th channel output received by the source-channel joint decoder is Where ω represents a differentiable noise function, which is used to implement backpropagation in model training; h represents the channel gain;

[0034]

[0035] Further, step S3 specifically includes: outputting the channel and the signal-to-noise ratio condition μ as the input of the source-channel joint decoder. First, Convert to real vector Then perform deconvolution operation θ: Get the initial estimated value y' of the point cloud feature, and finally pass y' through the coordinate reconstruction layer ψ: Get the initial estimate p' of the point cloud coordinates.

[0036] Further, step S4 specifically includes the following steps:

[0037] S41: Based on the point cloud coordinates and initial feature estimates, the self-attention mechanism is used to obtain new features of the input point cloud;

[0038] S42: Expand the signal-to-noise ratio condition μ to the same dimension as the point cloud feature and perform element-by-element addition;

[0039] S43: Calculate the scaling factor S based on the point cloud context information;

[0040] S44: Multiply the input point cloud features by the scaling factor element-by-element to obtain the point cloud attention features.

[0041] Furthermore, step S41 specifically includes: applying the maximum pooling operation to p' to obtain the global feature, copying the global feature rN times, and connecting it with the initial feature y' to obtain the new feature f global , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected with f through residual connections. global Add up to get the output point features

[0042] Furthermore, the training strategy for supporting multiple compression rates in step S5 includes the following steps:

[0043] S51: Define an available bandwidth ratio range ρ l ∈(0,0.05), where ρ l =k / C in N, k represents the dimension of the channel input symbol, C in Represents the dimension of the original input point cloud, and N represents the total number of points in the point cloud;

[0044] S52: After the steps S51 to S52 of the channel condition adaptive perception mechanism design, the receiving end bandwidth resource ρ l , perform a mask operation on the features and make an initial estimate of the masked features and their coordinate values;

[0045] S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set

[0046]

[0047] S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, the calculation expression of the distortion loss under the i-th compression rate is:

[0048]

[0049] S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function

[0050]

[0051] Among them, α and β are positive numbers to ensure that when When large, assign large weight Given the current bandwidth ratio ρ l ;Γ is the maximum weight;

[0052] That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, allocate the larger Give the loss under the current compression rate; otherwise reduce the corresponding weight

[0053] Furthermore, in step S6, the implementation steps of the deep source-channel joint decoder model are as follows:

[0054] S61: Apply the maximum pooling operation to the initial estimated value y' of the point cloud feature obtained after step S3 and copy it rN times, and then connect it with the initial estimated value y' of the point cloud feature to obtain a new feature

[0055] S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features

[0056] S63: f attention With f global Add element by element to get the output point features Pass F' through the adaptive module to obtain the encoded features

[0057] S64: The feature vector F obtained by the adaptive module A Input feature expansion module to obtain expanded point features Where r' represents the upsampling rate; the feature expansion module first replicates the input features r' times, and uses the transposed convolution branch to expand the features, learning new point features in a learnable way; finally, the point features from the two branches are concatenated and fed into a set of MLPs to generate expanded features:

[0058] F up =MLP(Concate[Dup(F A ,r');Deconv(F A )])

[0059] S65: Using residual learning strategy to expand the feature F up Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud:

[0060] ΔP=MLP(F up )

[0061]

[0062] The beneficial effect of the present invention is that: the present invention designs an adaptive deep source-channel joint coding method for wireless transmission of three-dimensional point clouds. When the object is a large amount of disordered point cloud data, the use of a method based on source-channel separation for encoding and wireless communication not only fails to achieve the optimal performance of the system through end-to-end optimization, but also faces the problem of the cliff effect in the communication system. During wireless channel transmission, the transmission of semantic information faces variable channel conditions. Due to the uneven distribution of point clouds, designing multiple networks for different signal-to-noise ratios will result in a significant increase in storage costs. In multi-user transmission scenarios, it is also necessary to consider the differences in bandwidth conditions at different receiving ends.

[0063] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0065] Figure 1 A flow chart of the system used in the present invention;

[0066] Figure 2 Schematic diagram of the channel condition adaptive sensing mechanism in step S4 of the present invention;

[0067] Figure 3 Schematic diagram of the training strategy for supporting multiple compression rates in step S5 of the present invention. DETAILED DESCRIPTION

[0068] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0069] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0070] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0071] like Figure 1 As shown, the present invention provides an adaptive depth source-channel joint coding method for wireless transmission of three-dimensional point clouds. The core of the method lies in the design of a channel condition adaptive perception mechanism and a multi-compression rate support scheme for multi-user transmission scenarios. The method mainly includes the following steps:

[0072] S1: Determine the deep source-channel joint encoder model, including the selection of the point cloud downsampling strategy and the design of the feature extraction module. Considering that the wireless transmission object is a large amount of disordered point cloud data, and the use of a method based on source-channel separation for encoding and wireless communication not only fails to achieve the optimal performance of the system through end-to-end optimization, but also faces the problem of cliff effect in the communication system, so the source-channel joint encoder model is adopted. The encoder model consists of a point cloud downsampling module using the farthest point sampling and a feature extraction module based on dynamic kernel convolution. The encoding end function is f θ : Where k represents the dimension of the channel input symbol. Specifically, the following steps are included:

[0073] S11: For a given reference point cloud Where N represents the total number of points in the point cloud, 3 represents the coordinate dimension of the point cloud, and the farthest point sampling strategy is adopted for P to obtain the downsampled point cloud Where r represents the downsampling rate; the farthest point sampling strategy steps are as follows:

[0074] S111: Randomly select a point from the point cloud as the first sampling point P0, add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud;

[0075] S112: For each unsampled point, calculate its minimum distance to all current sampling points, and select the farthest point P1 as the next sampling point;

[0076] S113: For all unsampled points, update their minimum distance to the sampling point set;

[0077] S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

[0078] S12: The downsampled point cloud P down The feature extraction module is used to capture rich semantic information and the spatial dynamic relationship of its geometric structure, denoted as C represents the dimension of the point cloud feature. The feature extraction module is implemented as follows:

[0079] Randomly initialize a basic convolution kernel B = {B m |m=1,...,M}, where Represents the weight matrix, M represents the number of weight matrices in the weight library B, C in with C out Represent the feature input and output dimensions respectively;

[0080] Position adaptive coefficient generation: Use K nearest neighbor algorithm to obtain point cloud P down The local neighborhood information is used to generate the position adaptive coefficient Score according to the position relationship between the center point and its neighboring points. ij =MLP(θ(p i ,p j )), where θ represents softmax normalization;

[0081] Dynamic convolution kernel generation: combined with weight matrix B m and its related position adaptation coefficient Score ij m Generate convolution weights

[0082] The adaptive convolution kernel enables the convolution operation on point cloud features to more effectively capture the spatial dynamic relationship on the point cloud geometric structure and capture rich semantic information.

[0083] Use the Point Transformer module to further extract point cloud neighborhood features:

[0084]

[0085] Among them, f i Represented as the i-th encoded point cloud feature vector, x i Represented as the feature vector of the i-th input point, q i Denote the corresponding query vector, k j Represented as x i The jth key value vector of the nearest neighbor, v jis represented as the j-th value vector, θ is represented as SoftMax normalization, α: represents the MLPs layer, δ: represents the position encoding function, ⊙ and They represent element-wise multiplication and addition respectively.

[0086] Figure 1 In the figure, TConv 128×128 and Conv 1×1 represent the transposed convolution and convolution operations and their convolution kernel sizes, respectively. C×128 and 128×3 represent the feature input and output dimensions, respectively. S128 and S1 represent the convolution step size.

[0087] S2: Select a power normalization method suitable for the unordered point cloud to obtain the complex semantic vector of the input channel It is transmitted on the noisy channel. According to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are Convert to latent variables And Normalize the power to meet the average power constraint Where ω represents a differentiable noise function, which is used to implement backpropagation in model training; h represents the channel gain. Specifically, the steps to implement the power normalization method are:

[0088] During the training phase, the sliding mean μ and variance σ of the point cloud are recorded.

[0089] In the inference phase, the (μ,σ) pair is used Normalize to get z' and convert it into a complex vector

[0090] The coded symbol z is transmitted over a noisy channel and is represented by the function η: The i-th channel output received by the source-channel joint decoder is Where ω represents a differentiable noise function, which is used to implement backpropagation in model training; h represents the channel gain.

[0091]

[0092] S3: Initially estimate the point cloud features and point cloud coordinates. First, the channel output Convert to real vector Then perform deconvolution operation θ: Get the initial estimated value y' of the point cloud feature, and finally pass y' through the coordinate reconstruction layer ψ: Get the initial estimate p' of the point cloud coordinates.

[0093] S4: As Figure 2As shown in the figure, a channel condition adaptive perception mechanism is designed, and the channel condition adaptation is achieved by using an element-by-element addition method. The specific implementation includes:

[0094] Using the element-by-element addition method, we first use the self-attention mechanism to obtain the new feature f of the input point cloud according to the point cloud coordinates and the initial feature estimate. i ', then expand μ to the point cloud feature f i 'Add the same dimension element by element to get the new feature vector It not only contains the original point cloud features, but also reflects the influence of signal-to-noise ratio conditions.

[0095]

[0096] Calculate the scaling factor S based on the context information:

[0097] S=sigmoid(s)

[0098] Multiply the input point cloud feature by the scaling factor element by element to obtain the point cloud attention feature F A :

[0099] F A =S i ·f i '

[0100] Optionally, in step S4, the self-attention mechanism is used to obtain the new feature f of the input point cloud based on the point cloud coordinates and the initial feature estimate. i ', specifically including: applying the maximum pooling operation to p' to obtain the global feature, copying the global feature rN times, and connecting it with the initial feature y' to obtain the new feature f global , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected with f through residual connections. global Add up to get the output point features

[0101] Figure 2 middle Denotes element-by-element addition, Conv 1×1 denotes the convolution operation and its convolution kernel size, 128×128 denotes the feature input and output dimensions, and S1 denotes the convolution step size.

[0102] S5: If Figure 3 As shown in the figure, a training strategy that supports multiple compression rates is designed. The following steps are included:

[0103] S51: Define an available bandwidth ratio range ρ l ∈(0,0.05), where ρ l =k / C inN, k represents the dimension of the channel input symbol, C in Represents the dimension of the original input point cloud, and N represents the total number of points in the point cloud.

[0104] S52: After the steps S1 to S2 of the channel condition adaptive perception mechanism design, the receiving end bandwidth resource ρ l , perform a simple masking operation on the features and make an initial estimate of the masked features and their coordinate values.

[0105] S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set

[0106] S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, the calculation expression of the distortion loss under the i-th compression rate is:

[0107]

[0108] S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function

[0109]

[0110] Among them, α and β are positive numbers to ensure that when When it is larger, assign a larger weight Given the current bandwidth ratio ρ l Γ is the maximum weight value, which is used to ensure stable training.

[0111] That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, a larger Give the loss under the current compression rate; on the contrary, it indicates that the current compression rate is more dominant and the corresponding weight needs to be reduced

[0112] S6: Determine the deep source-channel joint decoder model, including the design of the feature extraction module, feature expansion module, and the selection of the point cloud coordinate reconstruction strategy. The implementation steps are as follows:

[0113] S61: First, apply the maximum pooling operation to the initial estimated value y' of the point cloud feature obtained after step S3 and copy it rN times, and then connect it with the initial estimated value y' of the point cloud feature to obtain a new feature

[0114] S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features

[0115] S63: f attention With f global Add element by element to get the output point features Pass F' through the adaptive module to obtain the encoded features

[0116] S64: The feature vector F obtained by the adaptive module A Input feature expansion module to obtain expanded point features Where r' represents the upsampling rate. The feature expansion module combines the advantages of replication-based and learning-based methods. First, the input features are replicated r' times. At the same time, the transposed convolution branch is used to expand the features and learn new point features in a learnable manner. Finally, the point features from the two branches are concatenated and fed into a set of MLPs to generate expanded features:

[0117]

[0118] S65: Using residual learning strategy to expand the feature F up Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud:

[0119] ΔP=MLP(F up )

[0120]

[0121] In the above embodiments, references to "this embodiment" in the specification indicate that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily refer to the same embodiment.

[0122] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the present invention are intended to encompass all such alternatives, modifications, and variations that fall within the broad scope of the appended claims.

[0123] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0124] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0125] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0126] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0127] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0128] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0129] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0130] The present invention can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above.

[0131] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. An adaptive depth source-channel joint coding method for wireless transmission of three-dimensional point clouds, characterized by: The following steps are involved: S1: Construct a deep source-channel joint encoder model, including a point cloud downsampling module and a feature extraction module; S2: Obtain the complex semantic vector of the input channel using a power normalization method suitable for unordered point clouds , so that it is transmitted on the noisy channel; in step S2, according to the source channel joint encoder model determined in step S1, the point cloud features output by the last layer of the encoder are Convert to latent variables , and Perform power normalization, where Expressed as the downsampling rate, It is represented by the total number of points in the point cloud, and k represents the dimension of the channel input symbol. The steps for implementing the power normalization method include: S21: Record the sliding average of the point cloud during the training phase and variance ; S22: Used in the inference phase right Normalization is performed to obtain and converted to a complex-valued vector ; Encoded symbols Transmitted over a noisy channel, the function Indicates that the channel number received by the source-channel joint decoder is The output is ,in Represented as a differentiable noise function, used to implement backpropagation in model training; Expressed as channel gain; S3: Initially estimate the point cloud features and point cloud coordinates, specifically including: outputting the channel SNR As the input of the source-channel joint decoder, we first Convert to real vector Then perform deconvolution operation Get the initial estimate of the point cloud features , and finally Reconstructing layers by coordinates Get the initial estimate of the point cloud coordinates ; S4: Design a channel condition adaptive sensing mechanism, which includes the following steps: S41: Based on the point cloud coordinates and initial feature estimates, the self-attention mechanism is used to obtain new features of the input point cloud; S42: Set the signal-to-noise ratio condition Expand to the same dimension as the point cloud features and perform element-by-element addition; S43: Calculate scaling factor based on point cloud context information ; S44: Multiply the input point cloud feature by the scaling factor element by element to obtain the point cloud attention feature; S5: Train the deep source-channel joint encoder model using a training strategy that supports multiple compression rates. S6: Construct a deep source-channel joint decoder model, including a feature extraction module, a feature expansion module, and a point cloud coordinate reconstruction module.

2. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S1 constructs a deep source-channel joint encoder model, specifically including the following steps: S11: For a given reference point cloud ,in Expressed as the total number of points in the point cloud, Represents the coordinate dimension of the point cloud. Use the farthest point sampling strategy to get the downsampled point cloud ,in It is expressed as the downsampling rate; S12: The downsampled point cloud The feature extraction module captures the semantic information and the spatial dynamic relationship of its geometric structure, denoted as , Represented as the dimension of point cloud features.

3. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 2, characterized in that: The farthest point sampling strategy steps in step S11 are as follows: S111: Randomly select a point from the point cloud as the first sampling point , add it to the sampling point set, and use the Euclidean distance to measure the distance between each point and the sampling point cloud; S112: For each unsampled point, calculate the minimum distance from it to all current sampling points and select the farthest point As the next sampling point; S113: For all unsampled points, update their minimum distance to the sampling point set; S114: Repeat steps S113 to S114 until the size of the sampling point set reaches a predetermined number.

4. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 2, characterized in that: The feature extraction module in step S12 is implemented by the following steps: Randomly initialize a basic convolution kernel ,in represents the weight matrix, Representation weight library The number of weight matrices in , and Represent the feature input and output dimensions respectively; Position adaptive coefficient generation: using K nearest neighbor algorithm to obtain point cloud The local neighborhood information is used to generate the position adaptive coefficient according to the position relationship between the center point and its neighboring points. ,in Represented as softmax normalization; Dynamic convolution kernel generation: combining weight matrices and its related position adaptation coefficient Generate convolution weights ; Use the Point Transformer module to further extract point cloud neighborhood features: in, Expressed as The encoded point cloud feature vector, Expressed as Input point feature vectors, Denote the corresponding query vector, Expressed as Neighbor's key-value vectors, Expressed as A vector of values, Expressed as SoftMax normalization, express layer, represents the position encoding function, and They represent element-wise multiplication and addition respectively.

5. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: Step S41 specifically includes: Apply the maximum pooling operation to obtain global features and copy the global features times, and with the initial features Connect to get new features , and then use the self-attention mechanism to refine the local context of the point cloud. Finally, the point cloud features encoded by the self-attention mechanism are connected with Add up to get the output point features .

6. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: The training strategy for supporting multiple compression rates in step S5 includes the following steps: S51: Define a range of available bandwidth ratios ,in , represents the dimension of the channel input symbol, represents the dimension of the original input point cloud, Expressed as the total number of points in the point cloud; S52: After the steps S51 and S52 of the channel condition adaptive perception mechanism design, the receiving end bandwidth resources are used to , perform a mask operation on the features and make an initial estimate of the masked features and their coordinate values; S53: Under the same signal-to-noise ratio conditions, determine the upper limit of the average reconstruction quality of different compression rates on the validation set ; S54: Update the loss between the reconstructed point cloud and the original point cloud at each compression rate, and record the average reconstruction quality on the validation set in the inference phase Among them, The calculation expression of distortion loss under a compression rate is: S55: According to and The difference Perform gradient updates on models that support multiple compression rates and dynamically adjust the weight values ​​of the loss function ; in, and A positive number to ensure that When large, assign large weight Give the current bandwidth ratio ; is the maximum weight; That is, the average reconstruction quality on the validation set during the training phase and When the difference is large, allocate the larger Give the loss under the current compression rate; otherwise reduce the corresponding weight .

7. The adaptive depth source-channel joint coding method for three-dimensional point cloud wireless transmission according to claim 1, characterized in that: In step S6, the implementation steps of the deep source-channel joint decoder model are as follows: S61: Initial estimation of the point cloud features obtained after processing in step S3 Apply the max pooling operation and copy times, and then compared with the initial estimate of the point cloud features Connect to get new features , C is the dimension of point cloud features; S62: Use the Point Transformer module to refine the local context of the point cloud and obtain the encoded point cloud features ; S63: and Add element by element to get the output point features ;Will Obtain encoded features through the adaptive module ; S64: The feature vector obtained by the adaptive module Input feature expansion module to obtain expanded point features ,in Expressed as upsampling rate; the feature expansion module first copies the input features At the same time, the transposed convolution branch is used to expand the features and learn new point features in a learnable way; finally, the point features from the two branches are connected and fed into a set of Thus, the extended features are generated: S65: Using residual learning strategy to extend features Perform point-by-point offset learning, refine the point cloud coordinates to achieve coordinate reconstruction, and generate a reconstructed point cloud: 。

Images