Unmanned platform sensing information transmission method under network limited condition
By building a semantic transmission model of the visual Transformer module, combining image encoding and channel prediction modules, the problem of image data transmission delay and bit error rate of the unmanned platform under network restricted conditions is solved, and efficient and reliable perceived information transmission is achieved.
Patent Information
- Application Number
- CN202510622137.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2025-08-12
AI Technical Summary
When the communication network is limited, the traditional communication method transmits complex original image data with a large delay and high bit error rate, which cannot meet the communication needs. The semantic coding design is difficult and the channel modeling is uncertain, resulting in poor communication effects.
Design the visual Transformer module to build a semantic transmission model, combine image encoding, channel prediction and image decoding modules, and adaptively adjust semantic features through channel state information to achieve efficient transmission of image data.
Achieve efficient and reliable perceived image data transmission under low bandwidth and high noise network conditions, improving the performance and adaptability of semantic communication.
Smart Images
Figure CN120475412A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of semantic communication technology, and in particular to a method for transmitting perception information of an unmanned platform under network restricted conditions. Background Art
[0002] When unmanned platforms operate within limited communication networks, traditional communication methods suffer from long latency and high bit error rates when transmitting large amounts of complex raw image data. These methods, however, cannot meet the communication needs of unmanned platforms. Semantic communication, on the other hand, offers superior transmission performance. In a typical semantic communication system, the core of semantic communication lies not in data communication but in communicating the meaning or content of the data—in other words, "understanding first, then transmission."
[0003] Compared to traditional communication systems, semantic communication introduces semantic coding, extracting semantic features from raw data through semantic coding. This is then transmitted over a communication channel to the receiver, which then interprets these features. As a result, semantic communication allows the sender to better understand the purpose and context of the communication, eliminating the transmission of redundant data and alleviating the problem of limited communication resources. Furthermore, the background knowledge inherent in semantic communication improves the effectiveness and accuracy of communication systems, particularly in suboptimal communication environments with limited bandwidth, low signal-to-noise ratios, or high bit error rates. The accumulated contextual information and individual communication objectives of both communicating parties facilitate intelligent error correction and recovery of received information in poor communication environments and even in the event of partial information loss, ensuring efficient and reliable information transmission.
[0004] Semantic coding exploits semantic redundancy in the source to extract the most important semantic features. This is a form of lossy compression. Designing optimal semantic coding methods for image data and exploring the limits of lossy compression are challenging. Furthermore, semantic communication requires channel modeling, which must be incorporated into the semantic communication model for joint training to ensure its effectiveness. However, the communication environment is highly uncertain, making it difficult to accurately represent it using a statistical channel model. Therefore, channel estimation is necessary to obtain information about the current channel state. Summary of the Invention
[0005] The present invention provides a method for transmitting perception information of an unmanned platform under network-restricted conditions, comprising: collecting initial perception image data; inputting the initial image data into a semantic transmission model deployed on the unmanned platform, and recovering target perception image data to complete the perception information transmission of the unmanned platform.
[0006] In the above scheme, the semantic transmission model includes an image encoding module, a channel prediction module and an image decoding module; the image encoding module and the channel prediction module are deployed at the transmitting end of the unmanned platform, and the image decoding module is deployed at the receiving end of the unmanned platform.
[0007] In the above scheme, the initial image data is input into the semantic transmission model deployed on the unmanned platform, and the target perception image data is restored, including: inputting the initial perception image data into the image encoding module to obtain the initial semantic features; fusing the initial semantic features with the historical channel state information features in the channel prediction module to obtain the fused semantic features; and sending the fused semantic features and the predicted channel state information output by the channel prediction module to the image decoding module after passing through a preset Rayleigh channel model to restore the target perception image data.
[0008] In the above scheme, the image encoding module is used to extract image features through the convolution layer based on the input image data and combine them with the output results of the multi-head attention to output the features of each scale in the image; the channel prediction module is used to output the predicted channel state information based on the historical channel state information; the image decoding module is used to decode the features of each scale in the image to restore the image data.
[0009] In the above scheme, the semantic transfer model is obtained through training, and the training process includes: based on the historical image data set and the historical channel state information, the semantic transfer model is trained until the loss function converges.
[0010] In the above scheme, the initial semantic features are fused with the historical channel state information features in the channel prediction module to obtain the fused semantic features, including: encoding the historical channel state information in the channel prediction module to obtain the historical channel state information features; scaling the historical channel state information features to meet the needs of subsequent feature fusion; and performing point-by-point multiplication of the scaled historical channel state information features with the initial semantic features to fuse the historical channel state information features with the initial semantic features.
[0011] In the above solution, both the image encoding module and the image decoding module include multiple convolutional layers, multiple downsampling layers, and multiple ViT modules. The structure of the image encoding module and the structure of the image decoding module are arranged symmetrically.
[0012] In the above scheme, within the channel prediction module, predicted channel state information is output based on historical channel state information, including: obtaining historical channel state information; inputting the historical channel state information into the embedding layer to form a low-dimensional dense tensor representation after splicing and superposition position encoding; after the low-dimensional dense tensor representation passes through the encoder and decoder of the channel prediction module, the predicted channel state information is output.
[0013] In the above scheme, within the image coding module, the initial perceptual image data is input into the image coding module to obtain the initial semantic features, including: inputting the initial perceptual image data into the convolution layer and downsampling to achieve preliminary feature extraction; adaptively capturing the content of the features through the multi-head attention mechanism in the ViT module, and forming the initial semantic features after linear transformation.
[0014] In the above solution, when the transmitter of the unmanned platform is running, historical channel status information is continuously collected.
[0015] The technical solution of the embodiment of the present invention has at least the following beneficial effects:
[0016] (1) A visual Transformer module was designed based on the characteristics of image data, and a semantic transmission model for images was constructed based on this, enabling unmanned platforms to achieve efficient and reliable perception image data transmission under network constraints such as low bandwidth and high noise.
[0017] (2) A channel prediction module is introduced into the semantic transmission model to mine features from historical channel state information and implement adaptive adjustment of semantic features based on channel state to adapt to unknown communication environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flowchart of a method for transmitting perception information of an unmanned platform under network-restricted conditions according to an embodiment of the present invention is schematically shown;
[0019] Figure 2 Schematically shows a framework diagram of a semantic transmission model according to an embodiment of the present invention;
[0020] Figure 3 Schematically shows a framework diagram of a ViT module according to an embodiment of the present invention;
[0021] Figure 4 The following schematically shows a framework diagram of a channel prediction module according to an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.
[0023] Figure 1 The flowchart of the method for transmitting perception information of an unmanned platform under network-restricted conditions according to an embodiment of the present invention is schematically shown.
[0024] Please refer to Figure 1 The specific process of the unmanned platform perception information transmission method under network restricted conditions in an embodiment of the present invention includes operations S110 to S120.
[0025] In operation S110 , initial perception image data is collected.
[0026] In operation S120 , the initial image data is input into a semantic transmission model deployed on the unmanned platform, and target perception image data is recovered to complete the perception information transmission of the unmanned platform.
[0027] In an embodiment of the present invention, the semantic transmission model includes an image encoding module, a channel prediction module and an image decoding module; the image encoding module and the channel prediction module are deployed at the transmitting end of the unmanned platform, and the image decoding module is deployed at the receiving end of the unmanned platform.
[0028] Furthermore, the image encoding module is used to extract image features through the convolution layer based on the input image data and combine them with the output results of the multi-head attention to output features of each scale in the image; the channel prediction module is used to output predicted channel state information based on historical channel state information; the image decoding module is used to decode the features of each scale in the image to restore the image data.
[0029] In an embodiment of the present invention, initial image data is input into a semantic transmission model deployed on an unmanned platform, and target perception image data is recovered, including: inputting the initial perception image data into an image encoding module to obtain initial semantic features; fusing the initial semantic features with historical channel state information features in a channel prediction module to obtain fused semantic features; and sending the fused semantic features and the predicted channel state information output by the channel prediction module to an image decoding module after passing through a preset Rayleigh channel model to recover the target perception image data.
[0030] The construction of the semantic transfer model is described in detail below.
[0031] Figure 2 The following schematically shows a framework diagram of a semantic transmission model according to an embodiment of the present invention.
[0032] In an embodiment of the present invention, the architecture of the semantic transfer model is constructed by combining the convolutional layer and the ViT module, such as Figure 2 As shown in Figure 1, the architecture consists of an image encoding module, an image decoding module, and a channel prediction module. Both the image encoding module and the image decoding module include multiple convolutional layers, multiple downsampling layers, and multiple ViT modules. The image encoding module and the image decoding module are arranged symmetrically. The channel prediction module includes an encoder and a decoder.
[0033] Figure 3 The following schematically shows a framework diagram of a ViT module according to an embodiment of the present invention.
[0034] In an embodiment of the present invention, within the image encoding module, the initial perceptual image data is input into the image encoding module to obtain initial semantic features, including: inputting the initial perceptual image data into the convolution layer and performing preliminary feature extraction after downsampling; adaptively capturing the content of the features through the multi-head attention mechanism in the ViT module, and forming initial semantic features after linear transformation.
[0035] For example, according to the characteristics of image data, this embodiment designs a Vision Transformer (ViT) module to achieve content-adaptive feature capture. The ViT module is as follows: Figure 3 The most important component of the ViT module is the multi-head self-attention mechanism, the processing principle of which is described above. Furthermore, the ViT module leverages the advantages of convolutional neural networks in image processing, using convolutional layers to extract image features. These features are then combined with the output of the multi-head attention to obtain features at various scales within the image.
[0036] Through the embodiments of the present invention, the Transformer model in natural language processing is introduced into the field of computer vision to design a Vision Transformer (ViT) architecture to achieve content-adaptive feature capture, thereby accurately extracting semantic features from image data.
[0037] Figure 4 The following schematically shows a framework diagram of a channel prediction module according to an embodiment of the present invention.
[0038] In an embodiment of the present invention, within a channel prediction module, predicted channel state information is output based on historical channel state information, including: obtaining historical channel state information; inputting the historical channel state information into an embedding layer to form a low-dimensional dense tensor representation after splicing and superposition position encoding; and outputting the predicted channel state information after the low-dimensional dense tensor representation passes through the encoder and decoder of the channel prediction module.
[0039] For example, Figure 4 As shown in , based on the historical channel state information (CSI), the encoder in the Transformer model is used to obtain the feature information contained in the historical CSI. Figure 4 As shown, at time t, the historical CSI of the previous n moments is The input is fed into the embedding layer for dimensionality conversion and concatenation. Positional encodings are then superimposed to form a unified low-dimensional dense tensor representation, which is then fed into the encoder for processing. The multi-head self-attention module in the encoder is an evolution of the single-head self-attention mechanism. In the single-head multi-attention mechanism, the query, key, and value are calculated for each input. Attention weights are allocated to the value through the convergence of the query and key to generate the final output, automatically focusing on important features. The multi-head self-attention mechanism further transforms the query, key, and value using several different sets of linear projections, which are then fed into the attention pool in parallel. Finally, the outputs of all attention pools are concatenated to produce the final output, enabling the combination of multiple types of features and avoiding excessive concentration of attention.
[0040] Furthermore, in addition to receiving the output of the encoder, the decoder also embeds and position-encodes the output of the previous moment as input, and captures the information in the output of the previous moment through the multi-head attention mechanism to obtain the causal relationship in the output. The output of the previous moment and the output of the encoder are processed by the decoder together to implement the cross-attention mechanism. The output of the decoder passes through a fully connected layer and a softmax activation function layer to output the predicted channel state information at time t+1. .
[0041] Please continue reading Figure 2 In the image encoding module, due to the poor ability of the ViT module to extract shallow features, the initial perceptual image input data is first extracted using two convolutional layers and two downsampling operations. After obtaining deeper features, they are input into three consecutive ViT modules for processing. The multi-head attention mechanism in the ViT module adaptively captures the content of the features, and after linear transformation, the initial semantic features are formed. The initial semantic features are point-by-point multiplied with the scaled historical CSI features extracted by the channel prediction module, so that the historical CSI and the initial semantic features are integrated, allowing the semantic features to be adaptively adjusted according to network conditions.
[0042] Furthermore, the fused semantic features are scaled and input into a pre-set Rayleigh channel model. The predicted channel state information output by the channel prediction module is also input into the pre-set Rayleigh channel model. In the Rayleigh channel model, the channel serves as an intermediate layer connecting the image encoding module and the image decoding module, enabling end-to-end training of the semantic transmission system. After the semantic features are transmitted through the channel, they are decoded by the image decoding module, gradually recovering the target perceived image data from the semantic feature decoding.
[0043] Furthermore, in an embodiment of the present invention, the semantic transfer model is trained based on the historical image dataset and the historical channel state information until the loss function converges.
[0044] Specifically, the constructed semantic transfer model is trained by inputting historical image datasets and historical channel state information. For example, the Cifar-10 image dataset is input, and the image data is processed by the convolutional layers and the ViT module. Historical channel state information data collected in a real-world communication environment is input and processed by the channel prediction module. The loss function used in model training is the peak signal-to-noise ratio (PSNR) between the input image data and the restored image data. When the loss function converges, the semantic transfer model is considered to have achieved both semantic encoding and decoding capabilities.
[0045] Through the embodiments of the present invention, a channel prediction module based on the Transformer model is designed to predict real-time channel state information based on historical channel state information. The channel state information is used to adaptively adjust the extraction of semantic features, thereby improving the performance of semantic communication under network-constrained conditions.
[0046] Based on the semantic transmission model constructed and trained above, the communication transmission of the semantic transmission model is described in detail below.
[0047] In an embodiment of the present invention, initial image data is input into a semantic transmission model deployed on an unmanned platform, and target perception image data is recovered, including: inputting the initial perception image data into an image encoding module to obtain initial semantic features; fusing the initial semantic features with historical channel state information features in a channel prediction module to obtain fused semantic features; and sending the fused semantic features and the predicted channel state information output by the channel prediction module to an image decoding module after passing through a preset Rayleigh channel model to recover the target perception image data.
[0048] Furthermore, the initial semantic features are fused with the historical channel state information features in the channel prediction module to obtain fused semantic features, including: encoding the historical channel state information in the channel prediction module to obtain historical channel state information features; scaling the historical channel state information features to meet the needs of subsequent feature fusion; and performing point-by-point multiplication of the scaled historical channel state information features with the initial semantic features to fuse the historical channel state information features with the initial semantic features.
[0049] It is understandable that when an unmanned platform needs to transmit a large amount of initial perception image data, the image encoding module on the transmitter side implements the initial semantic features of the initial perception image data. At the same time, during the operation of the transmitter side, the channel prediction module continuously collects historical channel state information and extracts historical CSI (Channel State Information, CSI) features and outputs predicted channel state information. The historical CSI features are fused with the initial semantic features to obtain fused semantic features, thus achieving adaptive adjustment of semantic features based on channel state. The fused semantic features here are also semantic communication features, which are used to send to the receiver side of the unmanned platform to complete the communication of the unmanned platform.
[0050] Furthermore, the transmitter sends the fused semantic features and predicted channel state information to the receiver of the unmanned platform. The image decoding module deployed at the receiver decodes the semantic communication features and recovers the target perception image data, enabling the unmanned platform to transmit perception information. It can complete the information transmission with a very small amount of content transmission and effectively combat network constraints.
[0051] Through the embodiments of the present invention, image data collected by the unmanned platform is transformed into semantic features by deploying an image encoding module on the platform's transmitter. These semantic features are then integrated with historical channel state information output by a channel prediction module to achieve adaptive adjustment of the semantic features based on the channel state. The semantic features are then transmitted to the receiver, where an image decoding module converts the semantic communication features into image data, completing the transmission of sensory information.
[0052] The above specific embodiments further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for transmitting perception information of an unmanned platform under network-restricted conditions, characterized in that: The method comprises: Collecting initial perception image data; The initial image data is input into a semantic transmission model deployed on an unmanned platform, and target perception image data is recovered to complete the perception information transmission of the unmanned platform.
2. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 1, characterized in that: The semantic transmission model includes an image encoding module, a channel prediction module and an image decoding module; The image encoding module and the channel prediction module are deployed at the transmitting end of the unmanned platform, and the image decoding module is deployed at the receiving end of the unmanned platform.
3. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 2, characterized in that: Inputting the initial image data into a semantic transmission model deployed on an unmanned platform and recovering target perception image data includes: Inputting the initial perception image data into the image encoding module to obtain initial semantic features; Fusing the initial semantic features with the historical channel state information features in the channel prediction module to obtain fused semantic features; The fused semantic features and the predicted channel state information output by the channel prediction module are respectively sent to the image decoding module after passing through a preset Rayleigh channel model to restore the target perception image data.
4. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 2, characterized in that: The image encoding module is used to extract image features through the convolution layer based on the input image data and combine them with the output results of the multi-head attention to output the features of each scale in the image; The channel prediction module is used to output predicted channel state information based on historical channel state information; The image decoding module is used to decode features of each scale in the image to restore image data.
5. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 1 or 4, characterized in that: The semantic transfer model is obtained through training, and the training process includes: Based on the historical image dataset and historical channel state information, the semantic transmission model is trained until the loss function converges.
6. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 3, characterized in that: The fusing the initial semantic features with the historical channel state information features in the channel prediction module to obtain fused semantic features includes: In the channel prediction module, encoding the historical channel state information to obtain historical channel state information features; Performing a scale transformation on the historical channel state information features to meet the requirements of subsequent feature fusion; The historical channel state information features after the scale transformation are multiplied point by point with the initial semantic features, so that the historical channel state information features are fused with the initial semantic features.
7. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 2, characterized in that: The image encoding module and the image decoding module both include multiple convolutional layers, multiple downsampling layers, and multiple ViT modules. The structure of the image encoding module and the structure of the image decoding module are arranged symmetrically.
8. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 4, characterized in that: In the channel prediction module, outputting predicted channel state information based on historical channel state information includes: Get historical channel status information; The historical channel state information is input into the embedding layer and then concatenated and superimposed on the position encoding to form a low-dimensional dense tensor representation; After the low-dimensional dense tensor representation passes through the encoder and decoder of the channel prediction module, the predicted channel state information is output.
9. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 3 or 7, characterized in that: In the image encoding module, inputting the initial perceptual image data into the image encoding module to obtain initial semantic features includes: Inputting the initial perception image data into the convolution layer and performing preliminary feature extraction after downsampling; The multi-head attention mechanism in the ViT module is used to adaptively capture the content of features, and the initial semantic features are formed after linear transformation.
10. The method for transmitting perception information of an unmanned platform under network-restricted conditions according to claim 2 or 3, characterized in that: When the transmitting end of the unmanned platform is in operation, the historical channel state information is continuously collected.