A spaceborne cooperative image semantic transmission and calculation integrated method and system

Abstract

This invention discloses a satellite-based collaborative image semantic transmission and computing method and system, belonging to the field of satellite communication and edge computing technology. Addressing the limitations of satellite-to-ground link bandwidth and single-node computing power bottlenecks in Earth observation satellite networks, this invention involves a sensing satellite performing content-based lightweight adaptive compression on the original image, generating a compressed image representation and metadata, which is then sent to an edge computing satellite. The edge computing satellite, based on the metadata, uses a hybrid expert network to perform feature recovery on the compressed image, subsequently inputting it into a semantic inference model to extract semantic information relevant to the generation task, and finally downloading the lightweight semantic information to the ground station. This invention, through decoupled collaboration of inter-satellite computing power, abandons the traditional full-frame pixel downlink, significantly reducing satellite-to-ground transmission bandwidth consumption while ensuring high-precision and robust semantic transmission, breaking through the data downlink bottleneck of limited satellite transit windows.

Description

Technical Field

[0001] This invention relates to the fields of satellite communication, edge computing and image processing technology, and specifically to a satellite-based collaborative image semantic transmission and computing integrated method and system. Background Technology

[0002] With the development of low Earth orbit (LEO) Earth observation satellite networks, the massive amounts of observation data generated by high-resolution onboard image sensors have put enormous bandwidth pressure on inter-satellite links (ISL) and extremely limited satellite-to-ground links (SGL). Due to the short satellite transit window, the traditional "full acquisition, direct download, and ground processing" mode has extremely high transmission latency, which cannot meet the needs of time-sensitive tasks such as disaster monitoring and ship tracking.

[0003] For example, Chinese invention patent application CN200810226721.9 discloses a low bit-rate spaceborne image compression processing system and method. This system includes a control module, a wavelet transform module, a CCSDS scanning encoding module, and a bitstream splicing module. The system and method divide image data into several 32×32 (or 64×64, 128×128) small image blocks, which are then fed into the wavelet transform module, scanning encoding module, and bitstream splicing module to obtain a compressed bitstream. The wavelet transform module, scanning encoding module, and bitstream splicing module operate in a pipelined manner between image blocks, while the scanning encoding module operates in a pipelined manner between bit planes. The image compression processing system and method of this invention achieves compression performance between JPEG2000 and EZW, and at low bit-rate compression, its performance is comparable to JPEG2000. It employs a decoding resynchronization mechanism to prevent the spread of errors during space data transmission, resulting in low power consumption. The extensive use of parallel computing and pipelined structures improves the chip's processing speed, meeting the requirements for high-compression of spaceborne images.

[0004] For example, Chinese invention patent application number CN201910976760.9 discloses a method for detecting ship targets in spaceborne optical remote sensing images based on a lightweight receptive field pyramid. It adopts a method of constructing a lightweight receptive field pyramid by introducing dilated convolution, and constructs a multi-scale feature fusion detection module based on the multi-scale features extracted from the receptive field pyramid. Under the condition of limiting the network size, it improves the adaptability to the characteristics of ship targets in optical remote sensing images.

[0005] For example, Chinese invention patent application number CN202210658290.3 discloses a semantic communication method for intelligent tasks, belonging to the field of artificial intelligence and communication fusion technology. Specifically, it includes: first, the sending device collects image data, extracts feature maps through a neural network, calculates the gradient of the feature maps using the activation values ​​of neurons corresponding to semantic concepts, and then averages the gradients to obtain the importance weights of the feature maps relative to the semantic concepts, thus ranking the semantic concepts and feature maps; using the obtained semantic relationships, the feature maps are cropped and compressed, retaining the feature maps with high importance weights, and the compressed feature maps are transmitted through a wireless channel; the receiving end demodulates the received signal, recovers the feature map data, and inputs it into a subsequent neural network to complete the intelligent task, and returns the result to the sending device.

[0006] Current technologies still focus on optimizing image pixel-level compression efficiency, fidelity, throughput, and hardware implementation complexity, centering on reconstructed image quality or full-image compression performance. They lack semantic importance differentiation mechanisms for downstream tasks such as disaster monitoring and ship identification, easily wasting limited downlink bandwidth on task-irrelevant backgrounds or low-value pixel details. The processing link mainly relies on the local processor, memory, and power budget of a single satellite. When facing continuous high-resolution imaging, multi-task concurrent inference, or complex deep models, it is still easily limited by single-node computing power bottlenecks, energy consumption constraints, and insufficient model generalization ability. Current technologies have shifted from traditional bit-level reliable transmission to semantic-level efficient transmission, but most solutions still focus on end-to-end semantic encoding / decoding or network-layer semantic content distribution, and have not yet fully solved the problems of task splitting, computing power coordination, inter-satellite link bandwidth scheduling, and robust recovery of key semantic features under high compression rates in multi-node satellite networks of "sensing satellite-computing satellite-ground station".

[0007] In summary, the current field of spaceborne image transmission urgently needs a new architecture that can break through the computing power bottleneck of a single node and overcome the robustness defects of traditional compression and existing integrated transmission and computing solutions, so as to achieve high-precision, low-latency semantic data downlink under limited bandwidth. To solve the above-mentioned problems of existing technologies, this invention provides a spaceborne collaborative image semantic transmission and computing integrated method and system. Summary of the Invention

[0008] To address the aforementioned technical problems in existing technologies, this invention provides a method and system for integrated image semantic transmission and computation via spaceborne collaboration.

[0009] The present invention adopts the following technical solution:

[0010] This invention provides a spaceborne collaborative image semantic transmission and computation method, comprising:

[0011] Step 1: Acquire the original image to be transmitted, and use the sensing satellite to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata;

[0012] Step 2: The sensing satellite sends the compressed image representation and metadata to the edge computing satellite via an inter-satellite link;

[0013] Step 3: After receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata to reconstruct the task-oriented feature representation;

[0014] Step 4: The edge computing satellite will reconstruct the task-oriented feature representation and input it into the semantic inference model to extract and generate semantic information related to the specific task;

[0015] Step 5: The edge computing satellite transmits semantic information as a payload to the ground station via the satellite-to-ground link.

[0016] Furthermore, in step 1, the step of using sensing satellites to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata includes:

[0017] The original image is divided into multiple non-overlapping image blocks;

[0018] For each image patch, the redundancy of the image patch under different downsampling factors is evaluated based on structural similarity;

[0019] The redundancy is compared with a preset redundancy threshold, and the maximum downsampling factor that meets the redundancy threshold is adaptively selected for each image block.

[0020] Based on the selected maximum downsampling factor, the corresponding image block is compressed to obtain a compressed image representation.

[0021] The downsampling factor corresponding to each image block is recorded as compression ratio information, and combined with the color calibration clues of the image to generate the metadata.

[0022] Furthermore, in step 1, the redundancy is represented by the following formula (1):

[0023] (1),

[0024] in, For image blocks In the downsampling factor Redundancy below, For downsampling function, To restore the compressed features to their original resolution size using an upsampling interpolation function, This is a function for calculating structural similarity.

[0025] Furthermore, in step 3, after receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata to reconstruct a task-oriented feature representation, including:

[0026] Feature recovery of compressed image representation is performed using a hybrid expert network architecture, which includes routers and multiple parallel expert subnetworks.

[0027] Extracting local pixel features from compressed images to represent different spatial regions;

[0028] Parse metadata to obtain global compression ratio information corresponding to each spatial region;

[0029] Local pixel features are jointly embedded with global compression ratio information to generate routing guidance signals;

[0030] By using routing guidance signals to control routers, expert subnetworks with different weights are dynamically assigned to the features of different spatial regions for feature enhancement and repair, and a task-oriented feature representation is reconstructed.

[0031] Furthermore, in step 3, the routing guidance signal and the expert sub-network weights are represented by the following equation (2):

[0032] (2),

[0033] in, This is the gating weight vector output by the router. The extracted local pixel feature vector, This is a feature vector containing global compression ratio information parsed from metadata. This indicates a feature concatenation operation. Here is the learnable weight matrix for the router. For the corresponding bias vector, Represents the normalized exponential function, and the gated weight vector. The expert subnetwork used to determine the activation of Top-k participates in feature enhancement and repair.

[0034] This invention also provides a spaceborne collaborative image semantic transmission and computing system, comprising:

[0035] The adaptive compression module, deployed on the sensing satellite, is used to acquire the original images to be transmitted, perform content-based adaptive compression on the original images, generate compressed image representations and corresponding metadata, and send them to the edge computing satellite via inter-satellite links;

[0036] The feature recovery module, deployed on the edge computing satellite, is used to receive compressed image representations and metadata, and then perform compressed sensing feature recovery on the compressed image representations based on the metadata to reconstruct task-oriented feature representations.

[0037] The semantic extraction module, deployed on the edge computing satellite, is equipped with a pre-trained deep learning semantic inference model. It is used to input the reconstructed task-oriented feature representation into the model for continuous reasoning and to extract and generate structured semantic information related to a specific task.

[0038] The satellite-to-ground transmission module, deployed on the edge computing satellite, encapsulates the generated semantic information into lightweight data packets, which are then used as payloads to replace the original pixel matrix and transmitted to ground station nodes via the satellite-to-ground link for final mission application.

[0039] Furthermore, the adaptive compression module includes:

[0040] An image partitioning unit is used to divide the original image into multiple non-overlapping networked image blocks;

[0041] The redundancy evaluation unit is equipped with a structural similarity algorithm to calculate the redundancy of each image patch under different downsampling factors K, using the following formula to calculate the redundancy of the image patch. In the downsampling factor Redundancy below: ,

[0042] in, For image blocks In the downsampling factor Redundancy below; The downsampling function; An upsampling interpolation function to restore compressed features to their original resolution size; This is a function for calculating structural similarity.

[0043] The factor selection unit is used to select the redundancy calculation results. With the preset redundancy threshold The comparison is performed, and the maximum downsampling factor that meets the redundancy threshold is adaptively selected for each image patch.

[0044] The compression processing unit is used to call the image resampling algorithm to compress the corresponding image block based on the selected maximum downsampling factor, so as to obtain the compressed image representation with reduced volume.

[0045] The metadata generation unit records the downsampling factor corresponding to each image patch as compression ratio information, and combines it with the color calibration cues of the image to package the two into structured metadata.

[0046] Furthermore, the feature recovery module includes a hybrid expert network architecture, which specifically includes:

[0047] The local feature extraction unit is used to extract local pixel feature vectors representing different spatial regions of the compressed image. ;

[0048] The metadata parsing unit is used to parse the received metadata and obtain the global compression ratio information feature vector corresponding to each spatial region. ;

[0049] The joint embedding unit is used to jointly embed local pixel feature vectors with global compression ratio information feature vectors, performing feature concatenation operations. Generate routing guidance signals;

[0050] Routers, controlled by routing guidance signals, calculate and output gating weight vectors through their internal networks. The gating weight vector is calculated using the following formula. :

[0051] ,

[0052] in, and These are the learnable weight matrix and bias vector in the router, respectively. It is a normalized exponential function;

[0053] Multiple parallel expert subnetworks are used to determine the gating weight vector. The top-k expert subnetworks are selected by selecting the highest scores. The activated expert subnetworks perform nonlinear mapping on the input features and use their respective gating weights to perform weighted summation on the output results. This is to perform targeted feature enhancement and repair on feature degradation caused by high compression rate (such as block artifacts and edge blurring), and reconstruct a task-oriented feature representation that contains rich high-frequency semantic details and directly adapts to the inference needs of downstream networks in the tensor dimension.

[0054] Furthermore, the semantic inference model deployed in the semantic extraction module is a deep learning-based object detection network, used to extract the target bounding box coordinates, category labels, and confidence scores as semantic information from the task-oriented feature representation.

[0055] Furthermore, the satellite-to-ground transmission module encapsulates semantic information into structured data packets or binary short frames, which are then transmitted to the ground station via the satellite-to-ground link. The amount of semantic information data has a dimensionality reduction rate of over 98% compared to the amount of data in the original image.

[0056] Compared with the prior art, the superior effects of the present invention are as follows:

[0057] 1. The satellite-based collaborative image semantic transmission and computing method and system described in this invention decouples computing tasks. Specifically, the redundancy assessment and adaptive downsampling compression tasks, which have minimal computational load and lightweight parameters, are deployed on the sensing satellite, which is limited by power consumption and computing power. Meanwhile, the hybrid expert feature recovery and deep network semantic inference tasks, which require high computing power, are offloaded and deployed on the edge computing satellite, which has abundant computing resources. Through this asymmetric computing power decoupling design, efficient collaborative transmission and computing are achieved between the sensing satellite and the edge computing satellite. This architecture not only breaks through the computing power bottleneck of a single sensing satellite independently performing complex semantic extraction, but also abandons the traditional full-frame image downlink mode, transmitting only the final structured semantic information, such as target bounding boxes and category labels, in the satellite-to-ground link. This achieves a very high data dimensionality reduction rate in complex task scenarios and effectively solves the data transmission bottleneck during the limited satellite transit window.

[0058] 2. The image semantic transmission and computing integrated method and system of spaceborne collaboration described in this invention creatively introduces a hybrid expert (MoE) feature recovery architecture guided by metadata. When faced with image degradation caused by high compression rates, it can adaptively repair key features based on the prior information provided by metadata, avoiding the phenomenon of significant feature occlusion or loss caused by global uniform compression. This greatly improves the accuracy and robustness of downstream target detection tasks. At the same time, the lightweight redundant judgment model deployed on the sensing satellite side has a very small number of parameters, breaking the computing power bottleneck of the sensing node.

[0059] 3. The satellite-borne collaborative image semantic transmission and computing integrated system described in this invention encapsulates semantic information into structured data packets or binary short frames by an underground transmission module and transmits them to the ground station via a satellite-to-ground link. The amount of semantic information data has an assisted dimensionality reduction rate relative to the amount of data in the original image. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the integrated image semantic transmission and computation method for spaceborne collaboration described in this invention;

[0061] Figure 2 This is an architecture diagram of the application scenario of the spaceborne collaborative image semantic transmission system in this embodiment of the invention;

[0062] Figure 3 This is a logical schematic diagram of the adaptive compression module on the sensing satellite side in an embodiment of the present invention. Detailed Implementation

[0063] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0064] Example

[0065] like Figures 1 to 2 As shown, the spaceborne collaborative image semantic transmission and computation integrated method includes:

[0066] Step 1: Acquire the original image to be transmitted, and use the sensing satellite to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata. The original image is acquired by the high-resolution onboard camera of the sensing satellite. The image format is uncompressed raw RAW data or image data after preliminary denoising. The sensing satellite does not perform deep learning-level complex feature extraction on the original image. Instead, it uses the image patch-based structural similarity index Measure (SSIM) evaluation method to quickly analyze the content redundancy of different regions of the image and adaptively determine the optimal compression ratio for each region accordingly.

[0067] Step 2: The sensing satellite transmits the compressed image representation and metadata to the edge computing satellite via an inter-satellite link. The inter-satellite link uses the Ka-band or laser communication link, which has a much higher transmission rate than the satellite-to-ground link and is not affected by atmospheric attenuation or weather conditions, making it suitable as an inter-satellite data relay channel. The compressed image representation and metadata are packaged into data frames that conform to the CCSDS (Consultative Committee for Space Data Systems) space data link protocol standard and reliably transmitted to the edge computing satellite via the inter-satellite link.

[0068] Step 3: After receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata to reconstruct a task-oriented feature representation. As an on-board edge computing node, the edge computing satellite is equipped with high-performance onboard processing units, such as heterogeneous computing platforms based on FPGA (Field-Programmable Gate Array) or radiation-hardened GPU (Graphics Processing Unit). The edge computing satellite utilizes a Mixture of Experts (MoE) architecture, combined with the compression ratio prior information carried in the metadata, to perform differentiated feature enhancement and repair on the compressed image, recovering a high-quality feature representation suitable for downstream semantic inference tasks.

[0069] Step 4: The edge computing satellite inputs the reconstructed task-oriented feature representation into the semantic inference model to extract and generate semantic information related to the specific task. The semantic inference model adopts a lightweight design and on-orbit deployment optimized deep learning object detection network, such as the YOLO (You Only Look Once) series, EfficientDet, or a detection network specifically designed for remote sensing images. The semantic inference model receives the restored feature representation as input and outputs structured semantic information, including but not limited to bounding box coordinates, class labels, confidence scores, and optional object attribute information (such as ship length, heading angle, etc.).

[0070] Step 5: The edge computing satellite transmits semantic information as a payload to the ground station via the satellite-to-ground link. Unlike the traditional method of transmitting compressed image pixel data to the ground for processing, this invention encapsulates the extracted structured semantic information into lightweight data packets for transmission. For example, in a megapixel-level remote sensing image, the traditional method requires transmitting several megabytes (MB) of image volume data, while the method of this invention only requires transmitting a few hundred bytes to a few thousand bytes of target feature description information. While ensuring that the accuracy of downstream tasks (such as ship identification and vehicle detection) remains at a high level, it achieves a data dimensionality reduction rate of over 98%, completely breaking through the data transmission bandwidth bottleneck of Earth observation satellites under limited transit windows.

[0071] Furthermore, in step 1, the step of using sensing satellites to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata includes:

[0072] The original image is divided into multiple non-overlapping image blocks;

[0073] For each image patch, the redundancy of the image patch under different downsampling factors is evaluated based on structural similarity;

[0074] The redundancy is compared with a preset redundancy threshold, and the maximum downsampling factor that meets the redundancy threshold is adaptively selected for each image block.

[0075] Based on the selected maximum downsampling factor, the corresponding image block is compressed to obtain a compressed image representation.

[0076] The downsampling factor corresponding to each image patch is recorded as compression ratio information, and metadata is generated by combining it with the color calibration cues of the image.

[0077] Furthermore, to address the issue of excessive background redundancy in high-resolution remote sensing images, step 1 of the sensing satellite does not involve complex feature extraction at the deep learning level. Instead, it employs a lightweight image patching strategy, such as... Figure 3 As shown, the specific method of adaptive compression is as follows: For each image patch, its structural similarity at low resolution is calculated as a redundancy measure, and the redundancy is expressed by equation (1):

[0078] (1),

[0079] in, For image blocks In the downsampling factor Redundancy below, For downsampling function, To restore the compressed features to their original resolution size using an upsampling interpolation function, This is a function for calculating structural similarity.

[0080] Based on the current available bandwidth of inter-satellite or satellite-to-ground links and the accuracy requirements of specific downstream tasks, this invention pre-sets or dynamically issues a redundancy threshold. The specific adaptive selection logic is as follows: ensuring that the redundancy Φ(I,K) of the image patch under the downsampling factor K is not less than the redundancy threshold. Under the premise of ensuring high compression ratio, for patches with a large number of homogeneous backgrounds (such as oceans and large areas of forests), the system matches them with the maximum downsampling factor that meets the condition, thus assigning a higher compression ratio. For patches containing complex targets (such as ports and urban roads), since their structural similarity will drop rapidly when downsampling is large, the system will automatically match them with a smaller downsampling factor to preserve details. After compression, not only is a compressed image representation generated, but metadata is also extracted in parallel. This metadata records the compression ratio, structural information, and color information of different patches with minimal overhead (accounting for only a very small proportion of the total data). Since the number of parameters of this adaptive compression module is only about 1.53M, it greatly meets the low power consumption and low computing power requirements of the front-end sensing satellite.

[0081] Furthermore, since different image regions undergo varying degrees of downsampling compression at the sensing satellite end, traditional single neural networks struggle to effectively handle such heterogeneous degradation features. In step 3, after receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata, reconstructing a task-oriented feature representation, including:

[0082] Feature recovery of compressed image representation is performed using a hybrid expert network architecture, which includes routers and multiple parallel expert subnetworks.

[0083] Extracting local pixel features from compressed images to represent different spatial regions;

[0084] Parse metadata to obtain global compression ratio information corresponding to each spatial region;

[0085] Local pixel features are jointly embedded with global compression ratio information to generate routing guidance signals;

[0086] By using routing guidance signals to control routers, expert subnetworks with different weights are dynamically assigned to the features of different spatial regions for feature enhancement and repair, and a task-oriented feature representation is reconstructed.

[0087] Furthermore, the routing guidance signal and the expert subnetwork weights are represented by equation (2):

[0088] (2),

[0089] in, This is the gating weight vector output by the router. The extracted local pixel feature vector, This is a feature vector containing global compression ratio information parsed from metadata. This indicates a feature concatenation operation. Here is the learnable weight matrix for the router. For the corresponding bias vector, Represents the normalized exponential function, and the gated weight vector. The expert subnetwork used to determine the activation of Top-k participates in feature enhancement and repair. This design allows regions that have experienced high compression ratios and severe information loss to call upon expert network combinations with stronger recovery capabilities, while low-compression regions only require lightweight processing, thus balancing computing power and avoiding the loss of details caused by excessive smoothing.

[0090] like Figure 2 As shown, the present invention also provides a spaceborne collaborative image semantic transmission and computing integrated system, comprising:

[0091] The adaptive compression module, deployed on the sensing satellite, is used to acquire the original images to be transmitted, perform content-based adaptive compression on the original images, generate compressed image representations and corresponding metadata, and send them to the edge computing satellite via inter-satellite links;

[0092] The feature recovery module, deployed on the edge computing satellite, is used to receive compressed image representations and metadata, and then perform compressed sensing feature recovery on the compressed image representations based on the metadata to reconstruct task-oriented feature representations.

[0093] The semantic extraction module, deployed on the edge computing satellite, is equipped with a pre-trained deep learning semantic inference model. It is used to input the reconstructed task-oriented feature representation into the model for continuous reasoning and to extract and generate structured semantic information related to specific tasks.

[0094] The satellite-to-ground transmission module, deployed on the edge computing satellite, encapsulates the generated semantic information into lightweight data packets, which are then used as payloads to replace the original pixel matrix and transmitted to ground station nodes via the satellite-to-ground link for final mission application.

[0095] Furthermore, the adaptive compression module includes:

[0096] An image partitioning unit is used to divide the original image into multiple non-overlapping networked image blocks;

[0097] The redundancy evaluation unit is equipped with a structural similarity algorithm to calculate the redundancy of each image patch under different downsampling factors K, using the following formula:

[0098] ,

[0099] in, For image blocks In the downsampling factor Redundancy below; The downsampling function; An upsampling interpolation function to restore compressed features to their original resolution size; This is a function for calculating structural similarity.

[0100] The factor selection unit is used to select the redundancy calculation results. With the preset redundancy threshold The comparison is performed, and the maximum downsampling factor that meets the redundancy threshold is adaptively selected for each image patch.

[0101] The compression processing unit is used to call the image resampling algorithm to compress the corresponding image block based on the selected maximum downsampling factor, so as to obtain a compressed image representation with reduced size.

[0102] The metadata generation unit records the downsampling factor corresponding to each image patch as compression ratio information, and combines it with the color calibration cues of the image to package the two into structured metadata.

[0103] Furthermore, the feature recovery module includes a hybrid expert network architecture, which specifically includes:

[0104] The local feature extraction unit is used to extract local pixel feature vectors representing different spatial regions of the compressed image. ;

[0105] The metadata parsing unit is used to parse the received metadata and obtain the global compression ratio information feature vector corresponding to each spatial region. ;

[0106] The joint embedding unit is used to jointly embed local pixel feature vectors with global compression ratio information feature vectors, performing feature concatenation operations. Generate routing guidance signals;

[0107] Routers, controlled by routing guidance signals, calculate and output gating weight vectors through their internal networks. Calculated using the following formula:

[0108] ,

[0109] in, and These are the learnable weight matrix and bias vector in the router, respectively. It is a normalized exponential function;

[0110] Multiple parallel expert subnetworks are used to determine the gating weight vector. The top-k expert subnetworks are selected by selecting the highest scores. The activated expert subnetworks perform nonlinear mapping on the input features and use their respective gating weights to perform weighted summation on the output results. This is to perform targeted feature enhancement and repair on feature degradation caused by high compression rate (such as block artifacts and edge blurring), and reconstruct a task-oriented feature representation that contains rich high-frequency semantic details and directly adapts to the inference needs of downstream networks in the tensor dimension.

[0111] Furthermore, the semantic inference model deployed in the semantic extraction module is a deep learning-based object detection network, used to extract the target bounding box coordinates, category labels, and confidence scores as semantic information from the task-oriented feature representation.

[0112] Furthermore, the satellite-to-ground transmission module encapsulates semantic information into structured data packets or binary short frames, which are then transmitted to the ground station via the satellite-to-ground link. The amount of semantic information data has a dimensionality reduction rate of over 98% compared to the amount of data in the original image.

[0113] This invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the invention as claimed. The scope of protection of this invention is defined by the appended claims.

Claims

1. A spaceborne collaborative image semantic transmission and computation integrated method, characterized in that, include: Step 1: Acquire the original image to be transmitted, and use the sensing satellite to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata; Step 2: The sensing satellite sends the compressed image representation and metadata to the edge computing satellite via an inter-satellite link; Step 3: After receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata to reconstruct the task-oriented feature representation; Step 4: The edge computing satellite will reconstruct the task-oriented feature representation and input it into the semantic inference model to extract and generate semantic information related to the specific task; Step 5: The edge computing satellite transmits semantic information as a payload to the ground station via the satellite-to-ground link.

2. The spaceborne collaborative image semantic transmission and computation method according to claim 1, characterized in that, In step 1, the process of using sensing satellites to perform content-based adaptive compression on the original image to generate a compressed image representation and corresponding metadata includes: The original image is divided into multiple non-overlapping image blocks; For each image patch, the redundancy of the image patch under different downsampling factors is evaluated based on structural similarity; The redundancy is compared with a preset redundancy threshold, and the maximum downsampling factor that satisfies the redundancy threshold is adaptively selected for each image block. Based on the selected maximum downsampling factor, the corresponding image block is compressed to obtain the compressed image representation; The downsampling factor corresponding to each image block is recorded as compression ratio information, and combined with the color calibration clues of the image to generate the metadata.

3. The spaceborne collaborative image semantic transmission and computation method according to claim 2, characterized in that, The redundancy is expressed by the following formula (1): （1）， in, For image blocks In the downsampling factor Redundancy below, For downsampling function, To restore the compressed features to their original resolution size using an upsampling interpolation function, This is a function for calculating structural similarity.

4. The spaceborne collaborative image semantic transmission and computation method according to claim 1, characterized in that, In step 3, after receiving the compressed image representation and metadata, the edge computing satellite performs compressed sensing feature recovery on the compressed image representation based on the metadata to reconstruct a task-oriented feature representation, including: Feature recovery of compressed image representation is performed using a hybrid expert network architecture, which includes routers and multiple parallel expert subnetworks. Extracting local pixel features from compressed images to represent different spatial regions; Parse metadata to obtain global compression ratio information corresponding to each spatial region; Local pixel features are jointly embedded with global compression ratio information to generate routing guidance signals; By using routing guidance signals to control routers, expert subnetworks with different weights are dynamically assigned to the features of different spatial regions for feature enhancement and repair, and a task-oriented feature representation is reconstructed.

5. The spaceborne collaborative image semantic transmission and computation method according to claim 4, characterized in that, The routing guidance signal and the expert subnetwork weights are represented by the following equation (2): （2）， in, This is the gating weight vector output by the router. The extracted local pixel feature vector, This is a feature vector containing global compression ratio information parsed from metadata. This indicates a feature concatenation operation. Here is the learnable weight matrix for the router. For the corresponding bias vector, Represents the normalized exponential function, and the gated weight vector. The expert subnetwork used to determine the activation of Top-k participates in feature enhancement and repair.

6. A spaceborne collaborative image semantic transmission and computation integrated system, applied to the spaceborne collaborative image semantic transmission and computation integrated method as described in any one of claims 1 to 5, characterized in that, include: The adaptive compression module, deployed on the sensing satellite, is used to acquire the original images to be transmitted, perform content-based adaptive compression on the original images, generate compressed image representations and corresponding metadata, and send them to the edge computing satellite via inter-satellite links; The feature recovery module, deployed on the edge computing satellite, is used to receive compressed image representations and metadata, and then perform compressed sensing feature recovery on the compressed image representations based on the metadata to reconstruct task-oriented feature representations. The semantic extraction module, deployed on the edge computing satellite, is equipped with a pre-trained deep learning semantic inference model. It is used to input the reconstructed task-oriented feature representation into the model for continuous reasoning and to extract and generate structured semantic information related to specific tasks. The satellite-to-ground transmission module, deployed on the edge computing satellite, encapsulates the generated semantic information into lightweight data packets, which are then used as payloads to replace the original pixel matrix and transmitted to ground station nodes via the satellite-to-ground link for final mission application.

7. The spaceborne collaborative image semantic transmission and computing integrated system according to claim 6, characterized in that, The adaptive compression module includes: An image partitioning unit is used to divide the original image into multiple non-overlapping networked image blocks; The redundancy evaluation unit is equipped with a structural similarity algorithm to calculate the redundancy of each image patch under different downsampling factors K, using the following formula: ， in, For image blocks In the downsampling factor Redundancy below; The downsampling function; An upsampling interpolation function to restore compressed features to their original resolution size; This is a function for calculating structural similarity. The factor selection unit is used to select the redundancy calculation results. With the preset redundancy threshold The comparison is performed, and the maximum downsampling factor that meets the redundancy threshold is adaptively selected for each image patch. The compression processing unit is used to call the image resampling algorithm to compress the corresponding image block based on the selected maximum downsampling factor, so as to obtain the compressed image representation with reduced volume. The metadata generation unit records the downsampling factor corresponding to each image patch as compression ratio information, and combines it with the color calibration cues of the image to package the two into structured metadata.

8. The spaceborne collaborative image semantic transmission and computing integrated system according to claim 6, characterized in that, The feature recovery module includes a hybrid expert network architecture, which specifically includes: The local feature extraction unit is used to extract local pixel feature vectors representing different spatial regions of the compressed image. ; The metadata parsing unit is used to parse the received metadata and obtain the global compression ratio information feature vector corresponding to each spatial region. ; The joint embedding unit is used to jointly embed local pixel feature vectors with global compression ratio information feature vectors, performing feature concatenation operations. Generate routing guidance signals; Routers, controlled by routing guidance signals, calculate and output gating weight vectors through their internal networks. Calculated using the following formula: ， in, and These are the learnable weight matrix and bias vector in the router, respectively. It is a normalized exponential function; Multiple parallel expert subnetworks are used to determine the gating weight vector. The top-k expert subnetworks are selected by selecting the highest scores. The activated expert subnetworks perform nonlinear mapping on the input features and use their respective gating weights to perform weighted summation on the output results. This is to perform targeted feature enhancement and repair on the feature degradation caused by high compression rate, and reconstruct a task-oriented feature representation that contains rich high-frequency semantic details and is directly adapted to the inference needs of downstream networks in the tensor dimension.

9. The spaceborne collaborative image semantic transmission and computing integrated system according to claim 6, characterized in that, The semantic inference model deployed in the semantic extraction module is a deep learning-based object detection network used to extract target bounding box coordinates, category labels, and confidence scores as semantic information from task-oriented feature representations.

10. The spaceborne collaborative image semantic transmission and computing integrated system according to claim 6, characterized in that, The satellite-to-ground transmission module encapsulates semantic information into structured data packets or binary short frames, which are then transmitted to the ground station via the satellite-to-ground link. The amount of semantic information data has a dimensionality reduction rate of over 98% compared to the amount of data in the original image.

Images