Token semantic importance matched low-bandwidth satellite image transmission system

Abstract

The invention provides a low-bandwidth satellite image transmission system for token semantic importance matching. The low-bandwidth satellite image transmission system comprises a sending end, an erasure channel and a receiving end, the sending end is a satellite load end and comprises a token module and an unequal difference protection coding module based on an expansion window fountain code, the token module is used for converting a satellite image into a source token sequence containing semantic hierarchies, and the unequal difference protection coding module is used for generating an anti-erasure coding token according to a discrete token sequence; the coding token is transmitted by an erasure channel; the receiving end comprises a decoding module, a multi-mode token recovery module and an image reconstruction module, the decoding module is used for decoding the coded token, the multi-mode token recovery module is used for repairing the lost token, and the image reconstruction module is used for reconstructing the satellite image according to the repaired token. According to the method, the core semantic fidelity, the anti-interference capability and the spectrum efficiency can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a low-bandwidth satellite image transmission system based on token semantic importance matching. Background Technology

[0002] With the rapid development of low-Earth orbit satellite constellation networking and high-resolution remote sensing technology, satellite imagery has been widely used in key areas such as land surveying, weather forecasting, disaster monitoring, and national defense security. A single image generated by a high-resolution satellite can generate tens to hundreds of megabytes of data. However, satellite-to-ground links generally suffer from inherent constraints such as scarce bandwidth resources, susceptibility to ionospheric scintillation, and high erase / error rates due to multipath interference. Achieving efficient and robust transmission of satellite imagery under low-bandwidth conditions has become a core technological bottleneck in the field of satellite communications.

[0003] Semantic communication, as a novel paradigm in 6G communication, offers a new path to solving the low-bandwidth transmission problem by extracting semantic features from data rather than transmitting raw data. Token communication (TokCom), a large-model-driven semantic communication framework, tokenizes high-dimensional image data into discrete semantic tokens through a pre-trained large model, transmitting only the token sequence containing core semantics, significantly reducing the amount of data transmitted. Furthermore, it leverages the mask prediction capabilities of the large model to achieve token-level fault tolerance, naturally adapting to low-bandwidth transmission requirements. However, existing TokCom frameworks assume all tokens have equal semantic importance and employ an equal error protection (EEP) strategy. This approach has significant drawbacks: the importance of core semantics in satellite images (such as terrain contours, target facilities, cloud structures, etc.) differs greatly from local detail semantics (such as texture noise, slight grayscale differences). Traditional equal protection strategies lead to the waste of limited channel resources on non-critical information, while the core semantics that truly determine the value of image applications face the risk of loss due to channel erasure, ultimately making it difficult to balance transmission efficiency and semantic fidelity under low bandwidth conditions. Summary of the Invention

[0004] The main objective of this invention is to propose a low-bandwidth satellite image transmission system based on token semantic importance matching, which can effectively improve core semantic fidelity, anti-interference capability, and spectral efficiency.

[0005] This invention is achieved through the following technical solution:

[0006] A low-bandwidth satellite image transmission system based on token semantic importance matching, comprising a transmitter, an erasure channel, and a receiver;

[0007] The transmitting end is the satellite payload end, which includes a tokenization module and a non-equal difference protection coding module based on extended window fountain code. The tokenization module is used to convert satellite images into source token sequences with semantic hierarchy, and the non-equal difference protection coding module is used to generate erasure-resistant coded tokens based on discrete token sequences. The coded tokens are transmitted by the erasure channel.

[0008] The receiver includes a decoding module, a multimodal token recovery module, and an image reconstruction module. The decoding module is used to decode the encoded token, the multimodal token recovery module is used to repair the lost token, and the image reconstruction module is used to reconstruct the satellite image based on the repaired token.

[0009] Furthermore, the tokenization module utilizes a tokenization component of a model with discrete semantic representation capabilities to map the preprocessed satellite image into a discrete source token sequence T=[T1,T2,…,T] of length L. L It employs a tail-to-toe discarding design, encouraging the Tokenizer component to focus core semantics on the head of the source token sequence, so that the semantic importance of a token is inversely proportional to its position index. Here, T... L Let L be the Lth source token, and each source token is taken from a known pre-trained codebook.

[0010] Furthermore, the unequal difference protection coding module generates the coded token according to the following steps:

[0011] Step S11: Initialize parameters including the number of windows n, the total number of source tokens L, and the coding redundancy ρ;

[0012] Step S12: Divide the source token sequence T according to the number of windows to obtain n nested extended windows. ,in, ;

[0013] Step S13: According to the probability distribution Sampling extended window index j, where, Represents probability distribution Extended window The probability of index i. Represents formal variables;

[0014] Step S14: Based on the total number of source tokens, use the robust soliton degree distribution to sample the coding degree d from the window. d different source tokens are randomly selected from the pool. The selected source tokens are XORed with fixed-point operations to generate encoded tokens, where j=1,2,…,n;

[0015] Step S15: Repeat steps S11 to S14 to continuously generate encoded tokens and output the generated codes to the erasure channel, wherein the number of encoded tokens m = L × ρ.

[0016] Furthermore, the decoding module uses a belief propagation algorithm to decode the received encoded token. After decoding, the unrecovered tokens are marked with a mask value to obtain an error recovery token sequence containing the mask. .

[0017] Furthermore, the multimodal token recovery module extracts the error recovery token sequence. The unmasked token is read, and the category label text prompts for satellite imagery are read. ,Will and Inputting the data into the MaskGIT model, the MaskGIT model uses a self-attention mechanism to perform iterative mask prediction using contextual clues from adjacent tokens and textual hints from category labels. In each round of mask prediction, the predicted values ​​of the mask tokens are updated, ultimately resulting in a corrected token sequence. .

[0018] Furthermore, the image reconstruction module modifies the token sequence using the inverse tokenization component of the model with discrete semantic representation capabilities. Perform a reverse mapping to correct the token sequence. Reconstructed as satellite imagery.

[0019] Furthermore, the model with discrete semantic representation capabilities includes a lightweight One-D-Piece model, a ViT-based tokenization model, a MAE-derived semantic encoding model, or a pre-trained large model specifically for remote sensing images.

[0020] Furthermore, the preprocessing of the satellite image includes uniformly cropping the satellite image and normalizing the pixel values ​​to the range of [0,1].

[0021] Furthermore, the tail token discarding design includes: randomly sampling a truncated length L from a uniform distribution. t Then only the first L of the original token sequence are retained. t One token is discarded, and the rest are dropped.

[0022] Furthermore, the satellite image transmission system includes low-Earth orbit satellite links, medium-Earth orbit satellite links, or satellite-to-ground or interplanetary links from deep space exploration satellites.

[0023] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0024] This invention includes a transmitter, a channel, and a receiver. The transmitter is a satellite payload and includes a tokenization module and a non-uniformity protection coding module based on extended window fountain codes. The tokenization module converts satellite images into a source token sequence containing semantic levels. The non-uniformity protection coding module generates erasure-resistant coded tokens based on the discrete token sequence. The coded tokens are transmitted via an erasure channel. The receiver includes a decoding module, a multimodal token recovery module, and an image reconstruction module. The decoding module decodes the coded tokens, the multimodal token recovery module repairs lost tokens, and the image reconstruction module reconstructs satellite images based on the repaired tokens. The transmitter's coded and fused tokens, combined with the receiver's multimodal token recovery module, form a semantic-level anti-interference dual mechanism, improving multiple semantic fidelity metrics such as Peak Signal-to-Noise Ratio (PSNR), Perceptual Similarity Indicator (LPIPS), and Semantic Fidelity Indicator (CLIP Score). The tokenization module converts satellite images into source token sequences containing semantic layers before encoding, extracting only the core semantics of the satellite images. This significantly reduces the amount of data transmitted compared to traditional compression schemes, effectively solving the transmission congestion problem caused by insufficient bandwidth for massive amounts of high-resolution satellite images. This achieves efficient data transmission, thereby improving the performance indicators of the satellite image transmission system, such as spectrum utilization, energy efficiency, and real-time transmission. This invention is adaptable to existing digital communication systems without requiring significant modifications to existing network hardware architecture, reducing deployment costs and compatibility difficulties. It can be flexibly adapted to various satellite application scenarios such as land surveying, weather forecasting, and disaster monitoring. Attached Figure Description

[0025] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Figure 1 This is a schematic diagram of the architecture of the present invention.

[0027] Figure 2 This is a trend chart showing the semantic importance of tokens in this invention.

[0028] Figure 3 This is a structural comparison diagram of the extended window fountain code of the present invention and the traditional EEP fountain code.

[0029] Figure 4 The graph shows a performance comparison between the present invention and the comparative scheme under different overhead conditions.

[0030] Figure 5 The graph shows a performance comparison between the present invention and a comparative scheme under different channel erasure probabilities. Detailed Implementation

[0031] The present invention will be further described below through specific embodiments.

[0032] like Figure 1As shown, a low-bandwidth satellite image transmission system based on token semantic importance matching includes a transmitter, an erasure channel, and a receiver.

[0033] The transmitting end is the satellite payload end, which includes a tokenization module and a non-uniform protection coding module based on extended window fountain codes. The tokenization module is used to convert satellite images into source token sequences with semantic hierarchy, and the non-uniform protection coding module is used to generate erasure-resistant coded tokens based on discrete token sequences. The coded tokens are transmitted by the erasure channel.

[0034] Specifically, satellite image transmission systems include low-Earth orbit satellite links, medium-Earth orbit satellite links, or satellite-to-ground or inter-satellite links of deep space exploration satellites, or actual satellite channel simulation modes containing Doppler frequency offset / phase noise, and inter-satellite cooperative transmission channel scenarios.

[0035] The tokenization module utilizes a tokenization component of a model with discrete semantic representation capabilities to map the preprocessed satellite image into a discrete sequence of source tokens. This model with discrete semantic representation capabilities includes a lightweight One-D-Piece model, a ViT-based tokenization model, a MAE-derived semantic encoding model, or a large pre-trained model specifically for remote sensing images. In this embodiment, a lightweight One-D-Piece model is used, utilizing its tokenizer component to map the preprocessed satellite image into a 1-dimensional discrete sequence of source tokens T=[T1,T2,…,T] of length L=256. L It employs a tail-to-toe discarding design, forcing the Tokenizer component to prioritize encoding core semantics into the head token, so that the semantic importance of a token is inversely proportional to its position index, such as... Figure 2 As shown, T L Let L be the Lth source token, and each source token is taken from a known pre-trained codebook of size 4096.

[0036] The tail token discarding design specifically involves randomly sampling a truncated length L from a uniform distribution during training. t Then only the first L of the original token sequence are retained. t One token is discarded, and the rest are dropped.

[0037] Each satellite image is labeled with a category tag. The preprocessing of the satellite images specifically includes uniformly cropping the satellite images and normalizing the pixel values ​​to the range of [0,1].

[0038] The Unequal Difference Protection Coding Module (EWF-UEP) based on Extended Window Fountain Code generates a coded token according to the following steps, wherein the structure of the Extended Window Fountain Code (UEP) is compared with that of the traditional EEP fountain code as follows: Figure 3 As shown;

[0039] Step S11: Initialize parameters including window count n=2, total number of source tokens L=256, and coding redundancy ρ=1.2;

[0040] Step S12: Divide the source token sequence T according to the number of windows to obtain two nested extended windows w1={T1~T 64 w2={T1~T 256}, where w1⊂w2, and the length of each nested extended window is determined through simulation. More specifically, simulations can be performed using extended windows of different lengths, and the length with the best performance can be selected.

[0041] Step S13: According to the probability distribution Sampling extended window index j, where, Represents probability distribution Extended window The probability of index i. These are formal variables and do not have specific meaning.

[0042] Step S14: Based on the total number of source tokens, sample the coding degree d using the Robust Soliton Degree Distribution (RSD) from the window. d different source tokens are randomly selected from the pool. The selected source tokens are XORed with fixed-point operations to generate encoded tokens, where j=1,2;

[0043] Step S15: Repeat steps S11 to S14 to continuously generate encoding tokens and output the generated encoding to the erasure channel. The number of encoding tokens m = L × ρ is used to ensure low redundancy and adapt to satellite channel resource constraints.

[0044] In this embodiment, the erasure probability of the erasure channel is p=0.5.

[0045] The receiving end is a ground station, which includes a decoding module, a multimodal token recovery module, and an image reconstruction module. The decoding module is used to decode the encoded token, the multimodal token recovery module is used to repair the lost token, and the image reconstruction module is used to reconstruct the satellite image based on the repaired token.

[0046] The decoding module uses the belief propagation (BP) algorithm to decode the received encoded tokens. After decoding, it marks the unrecovered tokens with a mask value to obtain an error recovery token sequence containing the mask. The initial mask value is 4096.

[0047] The multimodal token recovery module extracts the error recovery token sequence. The unmasked token is read, and the category label text prompts for satellite imagery are read. ,Will and The input is the MaskGIT model, which uses a self-attention mechanism to perform iterative mask prediction using contextual clues from neighboring tokens and category text hints. In each round of mask prediction, the predicted values ​​of the mask tokens are updated, ultimately resulting in a corrected token sequence. .

[0048] The image reconstruction module corrects the token sequence using the inverse tokenization component of a model with discrete semantic representation capabilities. Perform a reverse mapping to correct the token sequence. Reconstructed as a satellite image. Specifically, in this embodiment, the token sequence will be corrected. Input the Detokenizer component of the lightweight One-D-Piece model. The Detokenizer component converts the token sequence through reverse mapping. Reconstructed as RGB image This completes one low-bandwidth satellite image transmission.

[0049] like Figure 4 As shown, when the coding redundancy ρ=1.2, the PSNR of the present invention is improved by 41.2%, 58.3%, and 44.7% respectively compared with the comparison schemes TokCom-EEP (token communication scheme with equal error protection), JPEG, and CompressAI, respectively; LPIPS is reduced by 42.6%, 49.1%, and 43.8% respectively; and CLIP score is improved by 11.5%, 26.4%, and 16.7% respectively, demonstrating the performance advantage of the present invention in low-overhead scenarios.

[0050] like Figure 5 As shown, within the channel erasure probability range of p=0.1~0.6, the spectral efficiency of the present invention (defined as "the ratio of the total number of original image pixels to the number of transmitted coding symbols") is higher than that of the TokCom-EEP scheme. When p=0.2, the spectral efficiency is 25.19% higher than that of TokCom-EEP, which verifies the resource utilization advantage of the present invention under low-bandwidth satellite links.

[0051] In this invention, the terms "first," "second," and "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. The use of terms such as "upper," "lower," "left," "right," "front," and "rear" to indicate orientation or positional relationships is based on the orientation or positional relationships shown in the accompanying drawings and is only for the convenience of describing the invention, not to indicate or imply that the device referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation on the scope of protection of this invention. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0052] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0053] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A low bandwidth satellite image transmission system with token semantic importance matching, characterized by: The system comprises a sending end, an erasure channel and a receiving end. The sending end is a satellite payload end, comprising a tokenization module and an unequal error protection coding module based on an extended window fountain code. The tokenization module is used for converting a satellite image into a source token sequence with semantic levels.

2. The token semantic importance matching low-bandwidth satellite image transmission system of claim 1, wherein: The tokenization module uses a tokenization component of a model with discrete semantic representation capabilities to map the preprocessed satellite image into a discrete source token sequence T=[T1,T2,…,T] of length L. L It employs a tail-to-toe discarding design, encouraging the Tokenizer component to focus core semantics on the head of the source token sequence, so that the semantic importance of a token is inversely proportional to its position index. Here, T... L Let L be the Lth source token, and each source token is taken from a known pre-trained codebook.

3. The token semantic importance matching low-bandwidth satellite image transmission system of claim 2, wherein: The unequal error protection coding module is used for generating anti-erasure coding tokens according to the discrete token sequence. The coding tokens are transmitted by the erasure channel. Step S12, dividing the source token sequence T according to the window number to obtain n nested extended windows wherein, ; Step S13, according to the probability distribution Sampling the extended window index i, where, is a formal variable, denotes the probability distribution of the index i of the extended window . Step S14, according to the total number of source tokens, a robust soliton degree distribution is used to sample the coding degree d, d different source tokens are randomly selected from the window The selected source tokens are combined with the fixed-point operation through the XOR operation to generate the coding token, j = 1, 2, …, n. The receiving end comprises a decoding module, a multi-modal token recovery module and an image reconstruction module.

4. The low bandwidth satellite image transmission system of claim 2 or 3, wherein: The decoding module decodes the received encoding token by using a belief propagation algorithm, and marks a mask value to a token which is not recovered after decoding, to obtain a sequence of error recovery tokens with mask .

5. The token semantic importance matching low-bandwidth satellite image transmission system of claim 4, wherein: The multi-modal token recovery module extracts an error recovery token sequence The unmasked tokens in the middle are not masked, and the category label text prompt of the satellite image is read , and The input MaskGIT model iteratively predicts the mask tokens by using the adjacent token context clues and the category label text prompt through the self-attention mechanism, and the prediction value of the mask token is updated in each round of mask prediction, and finally the corrected token sequence is obtained .

6. The token semantic importance matching low-bandwidth satellite image transmission system of claim 5, wherein: The image reconstruction module reconstructs the revised token sequence into a satellite image by means of an inverse tokenization component of the model with discrete semantic representation capability reconstructs the revised token sequence into a satellite image by means of an inverse tokenization component of the model with discrete semantic representation capability​ 7. The low bandwidth satellite image transmission system of claim 2 or 3, wherein: The decoding module is used for decoding the coding tokens.

8. The low bandwidth satellite image transmission system of claim 2 or 3, wherein: The multi-modal token recovery module is used for repairing lost tokens.

9. The low bandwidth satellite image transmission system of claim 2 or 3, wherein: The tail token dropping design includes: randomly sampling a truncation length L from a uniform distribution t Then only keep the first L t tokens of the original token sequence and drop the rest.

10. The low bandwidth satellite image transmission system of claim 1 or 2 or 3, wherein: The image reconstruction module is used for reconstructing a satellite image according to the repaired tokens. The unequal error protection coding module generates the coding tokens according to the following steps: Step S11: initializing parameters including a window number n, a total number of source tokens L and an encoding redundancy ρ. Step S15: repeating steps S11 to S14 to continuously generate coding tokens, and outputting the generated coding to the erasure channel. The number of coding tokens m is L×ρ. The model with discrete semantic representation capability comprises a lightweight One-D-Piece model, a ViT-based tokenization model, an MAE-derived semantic encoding model or a remote sensing image special pre-training large model. The preprocessing of the satellite image comprises uniformly cropping the satellite image and normalizing pixel values to the range of [0, 1]. The satellite image transmission system comprises a low-orbit satellite link, a medium-orbit satellite link or a star-ground or interstellar link of a deep space exploration satellite.

Images