Remote sensing image data compression and transmission method, terminal equipment and computer program product
By introducing attention mechanism and combined source channel coding technology in remote sensing image data transmission, the transmission performance problems in semantic information redundancy and variable signal-to-noise ratio environment are solved, and efficient and robust remote sensing image data compression and transmission.
Patent Information
- Application Number
- CN202411341582.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art lacks a mechanism for selectively retaining important features based on downstream tasks, resulting in redundancy of semantic information and degradation of transmission performance in variable signal-to-noise ratios and unstable communication environments.
Adopt attention-based important feature selection network and joint source channel encoding technology, and realize remote sensing image data through semantic feature extraction module, important feature selection module, joint source channel encoding module and decoder, combined with neural network and channel encoding. efficient compression and robust transmission.
It effectively reduces the amount of data transmitted, improves the completion quality of downstream tasks, enhances the robustness of the model, adapts to different signal-to-noise ratios and harsh environments, and realizes high-quality remote sensing image data compression transmission.
Smart Images

Figure CN119094772A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of semantic communication and compression technology, in particular to a remote sensing image data compression transmission method, terminal equipment and computer program product. Background Art
[0002] Remote sensing is the science of collecting information about an object without the need for direct physical contact. It involves the use of instruments such as satellites, aircraft or drones to capture data about the Earth's surface. These instruments produce airborne or satellite images of varying resolutions using techniques such as multi / hyperspectral and synthetic aperture radar observations. Advances in satellite imaging technology and the growing deployment of remote sensing instruments have led to an increasing use of remote sensing imagery. Understanding and classifying remote sensing images is essential for a variety of applications such as natural disaster detection, early warning systems, rescue missions, LULC determination, geospatial object detection, both for military and civilian purposes.
[0003] However, in a wireless communication environment, transmitting sensor data from remote locations (such as satellites, planets, and aircraft) to the ground faces multiple challenges. The first is the limited communication bandwidth: Due to interference from different spectra and rays in wireless communication, the available wireless bandwidth is highly restricted; the second is the complex wireless communication environment: The wireless communication process is constantly exposed to noise and interference from various sources; the third is the fragile communication link: The constant movement of acquisition equipment such as drones, satellites, and radars leads to frequent link switching and connection resets, further limiting the communication window; in addition, the problems associated with large amounts of data cannot be ignored, and it is not feasible to transmit all the large amounts of data.
[0004] In order to solve the problem of limited communication bandwidth and large data volume, source coding technology (compression coding scheme of source data or source signal) is used for data compression. At present, there are mainly two source coding methods.
[0005] 1. Traditional compression schemes: These schemes include lossless and lossy compression techniques. Lossless compression techniques like PNG and TIFF can perfectly reconstruct the original data from the compressed version. Lossy compression techniques, such as JPEG and BMP, achieve higher compression ratios by discarding less important information that cannot be fully recovered. However, traditional methods usually have low compression ratios and are prone to distortion because they cannot exceed the Shannon limit and have high bit error rates.
[0006] 2. Schemes based on semantic communication: These schemes combine semantic communication theory with neural networks, focus on the meaning and content of information, and emphasize the underlying semantics, thereby transforming raw data into more concise and meaningful information representations. By understanding and extracting semantics, these methods significantly reduce the amount of data transmitted. However, existing semantic communication methods lack a mechanism to selectively retain important features based on downstream tasks, resulting in semantic information redundancy, which needs further optimization.
[0007] To address fragile communication links and harsh wireless communication environments, the leading approach is joint source channel coding (JSCC): JSCC combines source and channel coding to improve communication reliability in noisy and interfering environments. Channel coding introduces redundancy in the original message, allowing correct reconstruction at the receiving end. Researchers have explored the use of deep neural networks inspired by autoencoders to design end-to-end JSCC structures, achieving high data recovery accuracy under fixed signal-to-noise ratios (SNRs) and a single noise type. However, the performance of JSCC depends heavily on the accuracy of the channel model, and most existing methods are trained based on fixed SNRs, resulting in performance degradation in variable and unstable environments. Maintaining performance across different SNRs requires training multiple models, which is complex and expensive. Summary of the invention
[0008] The technical problem to be solved by the present invention is to provide a remote sensing image data compression transmission method, terminal equipment and computer program product in view of the shortcomings of the existing technology, so as to solve the problems that the existing technology lacks a mechanism for selectively retaining important features based on downstream tasks, resulting in semantic information redundancy, and transmission performance degradation in the face of variable signal-to-noise ratio and unstable communication environment during communication.
[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: a remote sensing image data compression transmission method, comprising the following steps:
[0010] S1. constructing a data set using remote sensing images, and randomly dividing a training set using the data set;
[0011] S2, using the training set to train a remote sensing image data compression transmission model;
[0012] The remote sensing image data compression transmission model includes a semantic feature extraction module, an important feature selection module, a joint source channel coding module, a decoder, and a downstream task module;
[0013] The semantic feature extraction module is used to extract semantic information of remote sensing image data;
[0014] The important feature selection module is used to screen the semantic information and remove redundant semantic information;
[0015] The joint source-channel coding module is used to encode the semantic information after removing the redundant semantic information, and to perform simulated noise interference on the encoded result;
[0016] The decoder is used to receive the encoding result after the simulated noise interference, and restore the received encoding result to the semantic information before the encoding;
[0017] The downstream task module is used to convert the output of the decoder into the final output result.
[0018] The present invention introduces an attention mechanism while using semantic feature extraction technology. This attention mechanism can give different attention weights to semantic features according to different tasks, thereby achieving the purpose of retaining important features according to downstream tasks and avoiding the semantic feature redundancy problem caused by the unified processing of semantic features in the prior art. At the same time, a joint source channel coding technology solution is used, and random mixed interference of multiple types of channels and multiple types of noise are introduced at the same time, which can solve the problems of variable signal-to-noise ratio in the channel transmission process and decreased transmission performance in an unstable communication environment.
[0019] On the one hand, the present invention utilizes a neural network to extract semantic features from the input original remote sensing image, which can better, effectively and compactly represent the practical features of the remote sensing image; on the other hand, it utilizes an attention-based important feature selection network to remove redundant semantic features and improve the quality of downstream task completion. At the same time, it adopts joint source channel coding, introduces different types of channels, random signal-to-noise ratios and other simulated complex communication environments during training, improves the robustness of the model, and achieves the optimal compression transmission effect.
[0020] The semantic feature extraction module includes a first convolution layer, a first maximum pooling layer and four regular convolution blocks connected in series, each of the regular convolution blocks includes a first residual block and a second residual block; the first convolution layer is connected to the first maximum pooling layer, and the first maximum pooling layer is connected to the first regular convolution block; for the first regular convolution block, the input of its second residual block is the output of the first residual block and the output of the first maximum pooling layer; for the i-th regular convolution block, the input of its first residual block is the output of the i-1th regular convolution block and the output of the first residual block in the i-1th regular convolution block, and the input of the second residual block is the output of the i-1th regular convolution block and the output of the first residual block in the i-1th regular convolution block; i=2,3,4.
[0021] The semantic feature extraction in the present invention adopts a neural network based on residual blocks and cross-layer connections. The neural network based on residual blocks and cross-layer connections can extract semantic information from the original data and effectively reduce the amount of transmitted data. The residual blocks and cross-layer connection mechanism can effectively alleviate the gradient explosion and gradient vanishing problems in deep neural networks, ensure the long-term stability of the semantic feature extraction network, and effectively solve the problem of limited communication bandwidth from the existing acquisition end to the ground end and the huge amount of data that needs to be transmitted.
[0022] The specific implementation process of the semantic feature extraction module extracting semantic information includes:
[0023] 1) Set the convolution kernel to K, the size of the convolution kernel to k_size, the stride to Stride, and use the first convolution layer and the first maximum pooling layer to reduce the input remote sensing image Among them, channel is the number of channels, width is the width of a single channel, and hight is the height of a single channel, all of which belong to the range of the natural number set R, and the initial feature extraction result is obtained;
[0024] 2) The convolution kernel of the 2D convolution layer of the residual block in the regular convolution block is set to K_2, the size of the convolution kernel is 2k_size, and the stride is 2Stride. The first residual block performs a 2D convolution operation on the initial feature extraction result and connects the first batch processing normalization layer and the first ReLU activation function. The first ReLU activation function connects a 2D convolution operation layer and the second batch processing normalization layer;
[0025] 3) The initial feature extraction result and the output of the second batch normalization layer are used as the input of the second residual block. The second residual block performs a 2D convolution operation on the input. The output of the 2D convolution operation is used as the input of the third batch normalization layer. The output of the third batch normalization layer is used as the input of the second ReLU activation function. The output of the second ReLU activation function is subjected to another 2D convolution operation, and the result is input into the fourth batch normalization layer.
[0026] 4) The output of the previous regular convolution block is used as the input of the current regular convolution block. In the current regular convolution block, the first residual block performs a 2D convolution operation on the output of the previous regular convolution block, and connects the first batch processing normalization layer and the first ReLU activation function. The first ReLU activation function connects a 2D convolution operation layer and the second batch processing normalization layer;
[0027] 5) The output of the previous regular convolution block and the output of the second batch normalization layer are used as the input of the second residual block. The second residual block performs a 2D convolution operation on the input. The output of the 2D convolution operation is used as the input of the third batch normalization layer. The output of the third batch normalization layer is used as the input of the second ReLU activation function. The output of the second ReLU activation function is subjected to another 2D convolution operation, and the result is input into the fourth batch normalization layer.
[0028] 6) Repeat steps 4) and 5) to complete the operations of the third and fourth regular convolution blocks to obtain semantic information Among them, C is the number of channels, M is the width of a single channel, and N is the height of a single channel, all of which belong to the natural number set range.
[0029] 11. The remote sensing image data compression and transmission method according to claim 1 is characterized in that the important feature selection module includes a global average pooling layer, a first fully connected layer, a gated activation function,
[0030] Sigmoid activation function, mask matrix generator network.
[0031] The joint source-channel coding module includes a source encoder and a channel encoder; the source encoder includes four convolutional layers connected in sequence, each convolutional layer is followed by a ReLU activation function; the channel encoder includes Gaussian channel and Rayleigh fading channel coding; the Gaussian channel and Rayleigh fading channel coding add random Gaussian noise and randomly generated Rayleigh fading coefficients to the output of the source encoder to obtain an interfering coding result.
[0032] The decoder includes four deconvolution layers connected in sequence, the first three convolution layers are each followed by a ReLU activation function, and the last deconvolution layer is followed by a Sigmoid activation function.
[0033] The downstream task module includes an average pooling layer and a fully connected layer connected in sequence, the output of the fully connected layer is used as the input of the argmax function, and the output of the argmax function is the final output result.
[0034] For the trained remote sensing image data compression and transmission model, the semantic feature extraction module, important feature selection module, and joint source channel coding module are deployed at the remote sensing data acquisition end, and the decoder and downstream task modules are deployed at the ground end.
[0035] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor and a computer program stored in the memory; characterized in that the processor executes the computer program to implement the steps of the above method.
[0036] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the above method are implemented.
[0037] Compared with the prior art, the present invention has the following beneficial effects:
[0038] (1) Since the semantic feature extraction in the present invention adopts a neural network based on residual blocks and cross-layer connections, the neural network based on residual blocks and cross-layer connections can extract semantic information from the original data and effectively reduce the amount of transmitted data. The residual block and cross-layer connection mechanism can effectively alleviate the gradient explosion and gradient vanishing problems existing in the deep neural network, ensuring that the semantic feature extraction network has long-term stability. At the same time, it can effectively solve the problem of limited communication bandwidth from the existing acquisition end to the ground end and the huge amount of data that needs to be transmitted;
[0039] (2) Since the method for retaining important semantic features based on attention enhancement in the present invention adopts an attention mechanism based on semantic communication for downstream tasks, the attention mechanism based on semantic communication focuses on important features, which can effectively identify semantic information that is effective for downstream tasks, and discard irrelevant details according to the requirements of downstream tasks, thereby removing redundant semantic features that affect task accuracy, improving the quality of downstream tasks, and providing effective feature input for them; it can effectively solve the problem of a large amount of redundant information and waste of transmission resources in the transmission process;
[0040] (3) Since the joint source channel coding technology of the present invention adopts a four-layer convolutional neural network, it will further reduce the transmission data and compress the semantic information in the future. At the same time, it adopts channel coding technology to introduce various types and sizes of random noise and fading interference, which can cope with the complex environment and variable signal-to-noise ratio of wireless communication, improve the robustness of the model, and enable a single training model to adapt to remote sensing applications in different signal-to-noise ratios and harsh environments, and can effectively solve the problems of fragile communication links and harsh wireless communication environments in wireless communication environments;
[0041] (4) The method of the present invention is universal and can be applied to different downstream tasks in various situations. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a flow chart of a method for high-quality compression transmission of remote sensing image data from a remote sensing information collector to the ground for downstream tasks provided by the present invention;
[0043] Figure 2 It is a schematic diagram of a model for high-quality compressed transmission of remote sensing image data between a remote sensing information collector and the ground for downstream tasks provided by the present invention;
[0044] Figure 3 It is a model schematic diagram of the semantic feature extraction network provided by the present invention;
[0045] Figure 4 It is a schematic diagram of a model of an important feature selection network based on an attention mechanism provided by the present invention;
[0046] Figure 5 It is a schematic diagram of a model of a joint source-channel coding module network provided by the present invention;
[0047] Figure 6 It is a schematic diagram of the model structure of the decoder and downstream task network provided by the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] Example 1
[0050] This embodiment provides a high-quality compression transmission method for remote sensing image data from a remote sensing information collector to the ground for downstream tasks. The remote sensing image collected by the remote sensing image collector is passed through a semantic feature extraction network to generate a semantic feature representation that is more compact and more practical than the original image; then, an attention enhancement mechanism based on semantic communication is implemented for downstream tasks to complete the selection of important semantic features, and redundant semantic features are removed to provide effective semantic feature input for the task module, thereby improving the quality of task completion; at the same time, a joint source channel coding technology is adopted, and a four-layer convolution technology is used to further compress the size of the semantic features and reduce the amount of transmitted data. In order to cope with the random noise, fading, and various complex communication environments faced in the actual wireless channel transmission process, a variety of noise fading interferences are introduced to improve the robustness of the model, so that a single training model can adapt to remote sensing applications under different signal-to-noise ratios and harsh environments; the decoder at the ground end decodes the received semantic features, and then inputs them into the downstream task module to complete the task and output the results.
[0051] Combine the following Figures 1 to 6 The present invention provides a method and system for high-quality compression transmission of remote sensing image data from a remote sensing information collector to the ground for downstream tasks.
[0052] like Figure 1As shown, an embodiment of the present invention provides a method for high-quality compression transmission of remote sensing image data between a remote sensing information collector and the ground for downstream tasks, comprising the following steps:
[0053] 1. Collect historical remote sensing images and divide them into training set, validation set and test set in the ratio of 5:1:4. Use gradient descent method to train remote sensing image data compression transmission model at the ground station. The loss function is defined as Where K represents a multi-classification task with K categories, and the probability distribution of the model output is The true label is y = (y1, y2, ..., y k ), log represents the natural logarithm. This loss function penalizes the model for reducing the predicted probability of the correct class and encourages the model to make the probability of the correct class as close to 1 as possible. The model iteratively updates its weights on the training set to minimize the loss of the training data, and evaluates the performance of the model on the validation set to avoid overfitting. Finally, its effectiveness is evaluated on the test set to ensure generalization to new, unseen data.
[0054] like Figure 2 As shown, the remote sensing image data compression transmission model from the remote sensing information collector to the ground for downstream tasks in an embodiment of the present invention includes five parts: a semantic feature extraction module, an important feature selection module, a joint source channel coding module, a decoder, and downstream tasks.
[0055] Specifically, the purpose of the semantic feature extraction module is to extract compact and meaningful semantic feature representations of remote sensing images. We add residual blocks to the design to solve the common gradient vanishing and network degradation problems in traditional deep convolutional networks. Figure 3 As shown in the figure, the semantic feature extraction module specifically includes a first convolution layer and a first maximum pooling layer, and four regular convolution blocks of the same structure, where each regular convolution block includes two residual blocks, and each block contains several stacked layers and a jump connection. These connections allow the gradient to pass directly through the network, solving the problem of gradient disappearance and explosion that may occur in deep networks. Its specific structure is as follows:
[0056] (1) The first convolutional layer and the first maximum pooling layer: Convolution and pooling are used to quickly reduce the size of the input image and provide initial feature extraction for the regular convolution block.
[0057] (2) The four identical regular convolution blocks all contain a first residual block and a second residual block; both the first residual block and the second residual block consist of two 2D convolutions, two batch normalizations, and a ReLU activation function. There is a jump connection between the first residual block and the second residual block; this jump connection allows the input of the first residual block to bypass the second residual block and be added to the output of the first residual block before being fed into the second residual block;
[0058] (3) Four regular convolution blocks are executed serially and finally the semantic feature representation of the image is obtained.
[0059] The semantic feature extraction module performs the following steps on remote sensing image data:
[0060] (1) Set the convolution kernel to K, the size of the convolution kernel to k_size, and the stride to Stride. Use the first convolution layer and the first maximum pooling layer to transform the input remote sensing image Rapidly reduce and provide initial feature extraction results for the specified convolutional blocks;
[0061] (2) Set the convolution kernel of the 2D convolution layer of the residual block in the regular convolution block to K_2, the size of the convolution kernel to 2k_size, and the stride to 2Stride. The first residual block performs a 2D convolution operation on the result of (1) and connects a batch normalization and a ReLU activation, and then connects a 2D convolution operation and a batch normalization.
[0062] (3) The result of (1) is added to the result of (2) and used as the input of the second residual block. The second residual block performs a 2D convolution operation on the results of (1) and (2) and connects a batch normalization and a ReLU activation, and then connects a 2D convolution operation and a batch normalization.
[0063] (4) Repeat steps (2) and (3) to complete the second, third, and fourth rule block operations;
[0064] (5) Obtain the results of semantic feature extraction
[0065] After the semantic feature extraction module, the input image Convert to semantic information This process is recorded as:
[0066]
[0067] Among them, θ s Denotes all trainable parameters. The symbol C denotes the number of channels, which indicates the number of features extracted. In neural network design, the output dimensions M and N are one thirty-second of the original input dimensions. This significant compression is achieved through multiple strided convolutional layers.
[0068] Specifically, the purpose of the attention-based important feature selection module is to remove redundant information in the semantic feature representation that is meaningless to downstream tasks, thereby improving the quality of downstream task completion and reducing resource waste in communication transmission. Figure 4 As shown in the figure, the attention-based important feature selection module includes three network structures: Squeeze, Excitation, and Selection. Squeeze is used to condense the semantic feature map into a global feature representation. Excitation obtains the weight of each channel feature based on the global feature representation generated by Squeeze. Selection selects the corresponding important semantic features to be retained based on the weight. Its specific structure is as follows:
[0069] (1) Squeeze: Its input is the semantic features extracted from semantic features in Where c = 1, 2, ..., C. In order to measure the importance of each channel, a global average pooling layer is used to perform global averaging from the channel C dimension to output the global representation of the feature. Compress the information of each channel into a value representing the global importance of the channel, calculated as:
[0070]
[0071] Through the squeeze operation, the spatial dimensions of the feature map are reduced to a single value, resulting in a more compact and globally representative feature representation for each channel.
[0072] (2) Excitation: Its input is Excitation includes the following transformations:
[0073]
[0074] The MLP consists of a hidden layer and an output layer with dimensions C / r and C, and consists of activation functions of ReLU and Sigmoid, respectively. In order to avoid introducing unnecessary drift, bias is not used here. r is a hyperparameter used to control the dimension of the intermediate layer during importance learning, balancing computational efficiency and importance capture capabilities. In this example, r=16 is fixed. θ mlp Representing all trainable parameters in MLP, Excitation captures the dependencies between channels and generates a set of channel weights It represents the importance of each channel feature in different tasks.
[0075] (3) Selection: The weights in W correspond to the importance of the C channel. In order to retain important features, Remove unimportant channels and select more important channels (or weighted channels). Use a mask matrix generator network to generate the mask matrix w′∈{0,1} C ;The following steps of the mask generator network are completed:
[0076] First, generate a mask vector w′=[w1′, w2′, ..., w c ′]∈{0,1} C , according to the weight W = [W1, W2, ..., W c ], where c = 1, 2, ..., C. The process of generating w′ is as follows: first calculate the ηCth largest value of W as the threshold τ (the number of features to be retained is determined by a hyperparameter η∈(0,1]) and set the value of w′ to 1 if the value of w is greater than or equal to τ, and set the value of w′ to 0 otherwise:
[0077]
[0078] Then perform matrix multiplication on the mask matrix and the original semantic features to output the semantic features after removing redundancy. This process can be formalized as:
[0079]
[0080] Where ⊙ represents the channel selection operation. If w c ′=1, then keep The cth channel of , and vice versa. The pruned feature map Contains the most important channel features and can be used for subsequent tasks. By adjusting the parameter η, the number of selected important features can be controlled and the original semantic information can be represented in a more compact form.
[0081] Specifically, the main purpose of the joint source channel coding module is to alleviate the problem of the bandwidth of the long-distance ground communication system remaining basically unchanged for a long time due to the limitation of spectrum resources, technical limitations and costs, and at the same time, considering the influence of the atmospheric environment (such as absorption, scattering, refraction), different weather conditions, different terrains and electromagnetic interference from other communication equipment (such as radar, TV signals, etc.), to improve the robustness of data transmission in the communication environment. Figure 5As shown, the joint source-channel coding module mainly includes two parts: a source coding module and a channel coding module. The source encoder contains four convolutional layers. The convolutional kernel is \(K_S\), the size of the convolutional kernel is \(KS\_size\), and the stride is \(KStride\). A ReLU activation function is connected after each convolutional layer. The specific structure of the joint source-channel coding module is as follows:
[0082] Using neural network-based coding techniques, the redundant semantic features are further compressed to reduce the transmission cost and effectively represent information. A four-layer convolutional network is used to compress the input semantic information into a compact code for more efficient transmission, where \(l < L\), \(m < M\), \(n < N\). The source coding module can be expressed as:
[0083]
[0084] where \(\theta\) c is the set of trainable parameters of the coding module. Convs represents a convolutional neural network with 4 layers.
[0085] The channel encoder includes Gaussian channel and Rayleigh fading channel coding. The Gaussian channel and Rayleigh fading channel coding add random Gaussian noise and the influence of randomly generated Rayleigh fading coefficients to the output of the source encoder. During the model training process, random noise and random fading interference are introduced, and at the same time, two different channel conditions, AWGN and Fading, are simulated to more accurately reflect the channel conditions in real transmission, so as to improve the robustness under different signal-to-noise ratios and complex interference environments during transmission. The first type: AWGN, a traditional additive white Gaussian noise channel. Random Gaussian white noise is added during the coding in each training epoch to simulate the real-time changing noise conditions in the real environment:
[0086]
[0087] where each element of AWGN noise follows \(N(0, P / \gamma)\), where \(P\) and \(\gamma\) represent the symbol power and the signal-to-noise ratio in linear scale, and the random range of SNR is set to 0 - 28 dB.
[0088] The second type is Fading. In each epoch, first generate a Fading channel coefficient which consists of The fading channel is composed of complex Gaussian random variables of the same size, and the coefficient of the fading channel is then used to determine the number of channels to be used. The input signal is then multiplied by the channel coefficient. Finally, a complex form of Gaussian white noise Ε is generated based on the randomly generated SNR (0-28dB) and added to the signal. The method of introducing fading is as follows:
[0089]
[0090] Among them, the channel coefficient is a complex-valued random variable whose real and imaginary parts follow an independent and identically distributed Gaussian distribution with zero mean and equal variance. Each element of is composed of ε0+j∈1, ε0 and ∈1 are independent zero-mean, Gaussian distributed random variables, ε0, ∈1~N(0,γ 2 ), j is the sign of the imaginary part. E is additive white Gaussian noise in complex form. As the data experiences channel fading, it is further affected by E.
[0091] Therefore, the ground end receives the semantic feature encoding with noise
[0092]
[0093] Specifically, the decoder and downstream task modules work on the ground end to encode the received semantic features. Decoding to remove redundant semantic features before compression And it is input as high-quality input to the downstream task module to obtain the final task result. Figure 6 As shown, its specific structure is as follows:
[0094] The decoder first applies a deconvolutional neural network to the received code Decode it and restore it to This process is recorded as:
[0095]
[0096] Among them, θ d is the trainable parameter set of the decoding module, and DeConvs is a deconvolutional neural network. DeConvs has a symmetrical structure relative to Convs and is a four-layer deconvolutional neural network.
[0097] It is worth noting that in traditional image compression scenarios, decoded semantic features are used to approximate the original image. However, in this case, data loss is indiscriminate. Therefore, unlike traditional image compression, in the embodiment of the present invention, the reconstructed semantic features are input into the downstream task for task-oriented supervised learning to ensure that the lost information is not important to the downstream task.
[0098] At this point, this embodiment sets the last task to perform scene classification of remote sensing images. Scene classification of remote sensing images is of great significance for environmental protection, agricultural management, military security and climate change research. The output of the decoding module will be directly input into the classification module.
[0099] Specifically, the task module receives the semantic information Perform average pooling to convert channel features into The feature map of:
[0100]
[0101] The feature map is then classified through a fully connected layer (FC) and a softmax function, and the classification result is obtained through an argmax:
[0102]
[0103] in, represents the probability corresponding to all categories, Indicates the final category.
[0104] 2. The model of high-quality compressed transmission of remote sensing image data from the remote sensing information collector of the trained complete downstream task to the ground is divided into five parts: semantic feature extraction module, important feature selection module, joint source channel coding module, decoder, and downstream task module. The semantic feature extraction module, important feature selection module, and joint source channel coding module are deployed on the remote sensing data acquisition equipment, the decoder and downstream task module are deployed on the ground, and the simulated channel for simulating the transmission medium during the training process is omitted.
[0105] 3. After the remote sensing data acquisition equipment collects the remote sensing image data, it passes through the semantic feature extraction module, the important feature selection module, and the source encoder in the joint source channel coding to obtain the encoding result. They extract and filter the semantic information of the remote sensing image and give it the ability to resist the noise of the communication channel, which can not only greatly save the bandwidth of the satellite-to-ground communication link, but also resist channel noise of different intensities; after the ground end receives the encoding result contaminated by noise, it is reconstructed into a semantic feature before compression to remove redundancy through the decoder, and then input into the downstream task module. The semantic features of the task module will generate the final task result. The performance of the deployed model is continuously monitored and routine maintenance is performed. This includes verifying the adaptability of the model to the changing data distribution and implementing updates or retraining as needed to ensure the effectiveness of ongoing operations.
[0106] Experimental Results
[0107] The experimental environment of this embodiment is: under Ubuntu 16.04 operating system, the CPU is Intel (R) Xeon (R) Gold5318Y, the GPU is 1 NVIDIA RTX 4090Ti 24GB, the memory is 512GB DDR4, and Pytorch programming is used to implement the algorithm of the present invention. The Adam random optimization method is used to update the model parameters, the batch size is 64, and the learning rate is 10 -4
[0108] The embodiment of the present invention evaluates the performance of compressed transmission on four widely used remote sensing image datasets NWPU-RESISC45, UCMercedLandUse, Aerial Image Dataset (AID), and RSSCN7 Dataset. The evaluation indicators used are classification accuracy ACC and compression ratio CR as evaluation indicators of task-oriented compressed transmission.
[0109] Classification accuracy ACC refers to the ratio of the number of samples (images) correctly classified by the model on the test data to the total number of samples. The specific formula is as follows:
[0110]
[0111] The compression ratio CR is a measure of the effectiveness of data compression, usually used to describe the reduction in model size, memory usage, or storage space. It is calculated as follows:
[0112]
[0113] Table 1 shows the classification accuracy ACC and compression ratio CR of NWPU-RESISC45 data under different channel conditions, including the remote sensing image data compression transmission method between the remote sensing information collector and the ground for downstream tasks provided by an embodiment of the present invention and 6 other classification and semantic communication algorithms and their variants.
[0114] Table 2 shows the classification accuracy ACC and compression ratio CR of the remote sensing image data between the remote sensing information collector and the ground for downstream tasks provided by an embodiment of the present invention and 6 other classification and semantic communication algorithms and their variants of the Aerial Image Dataset (AID) data under different channel conditions.
[0115] Table 3 shows the classification accuracy ACC and compression ratio CR of UCMerced LandUse data between the remote sensing information collector and the ground for downstream tasks provided by an embodiment of the present invention and 6 other classification and semantic communication algorithms and their variants under different channel conditions.
[0116] Table 4 shows the classification accuracy ACC and compression ratio CR of the remote sensing image data between the remote sensing information collector and the ground for downstream tasks provided by an embodiment of the present invention and 6 other classification and semantic communication algorithms and their variants of the RSSCN7 Dataset data under different channel conditions.
[0117] Table 1 Classification accuracy ACC and compression ratio CR of NWPU-RESISC45 data under different channel conditions by different algorithms
[0118]
[0119] Table 2 Classification accuracy ACC and compression ratio CR of Aerial Image Dataset (AID) data under different channel conditions by different algorithms
[0120]
[0121] Table 3 Classification accuracy ACC and compression ratio CR of UCMerced LandUse data under different channel conditions by different algorithms
[0122]
[0123]
[0124] Table 4 Classification accuracy ACC and compression ratio CR of RSSCN7 Dataset data under different channel conditions by different algorithms
[0125]
[0126] According to the experimental results of Tables 1 to 4 under four data sets, it can be seen that the remote sensing image data compression transmission method between the remote sensing information collector and the ground for downstream tasks provided by the embodiment of the present invention is superior to the existing method in two different channel states of AWGN and Fading and two indicators of ACC and CR.
[0127] The embodiment of the present invention provides a method for compressing and transmitting remote sensing image data from a remote sensing information collector to the ground for downstream tasks. On the one hand, a neural network is used to extract semantic features of the input original remote sensing image, which can better, effectively and compactly represent the practical features of the remote sensing image; on the other hand, an attention-based important feature selection network is used to remove redundant semantic features, thereby improving the quality of downstream task completion. At the same time, joint source channel coding is adopted, and different types of channels, random signal-to-noise ratios, etc. are introduced during training to simulate complex communication environments, thereby improving the robustness of the model and achieving the optimal compression transmission effect.
[0128] Example 2
[0129] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.
[0130] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.
[0131] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.
[0132] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.
[0133] Example 3
[0134] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above embodiment 1 are implemented.
[0135] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.
[0136] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0137] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0139] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0140] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A remote sensing image data compression and transmission method, characterized in that: The following steps are involved: S1. constructing a data set using remote sensing images, and randomly dividing a training set using the data set; S2, using the training set to train a remote sensing image data compression transmission model; The remote sensing image data compression transmission model includes a semantic feature extraction module, an important feature selection module, a joint source channel coding module, a decoder, and a downstream task module; The semantic feature extraction module is used to extract semantic information of remote sensing image data; The important feature selection module is used to screen the semantic information and remove redundant semantic information; The joint source-channel coding module is used to encode the semantic information after removing the redundant semantic information, and to perform simulated noise interference on the encoded result; The decoder is used to receive the encoding result after the simulated noise interference, and restore the received encoding result to the semantic information before the encoding; The downstream task module is used to convert the output of the decoder into the final output result.
2. The remote sensing image data compression and transmission method according to claim 1, characterized in that: The semantic feature extraction module includes a first convolution layer, a first maximum pooling layer and four regular convolution blocks connected in series, each of the regular convolution blocks includes a first residual block and a second residual block; the first convolution layer is connected to the first maximum pooling layer, and the first maximum pooling layer is connected to the first regular convolution block; for the first regular convolution block, the input of its second residual block is the output of the first residual block and the output of the first maximum pooling layer; for the i-th regular convolution block, the input of its first residual block is the output of the i-1th regular convolution block and the output of the first residual block in the i-1th regular convolution block, and the input of the second residual block is the output of the i-1th regular convolution block and the output of the first residual block in the i-1th regular convolution block; i=2,3,4.
3. The remote sensing image data compression and transmission method according to claim 2, characterized in that: The specific implementation process of the semantic feature extraction module extracting semantic information includes: 1) Set the convolution kernel to K, the size of the convolution kernel to k_size, the stride to Stride, and use the first convolution layer and the first maximum pooling layer to reduce the input remote sensing image Get the initial feature extraction result; where channel is the number of channels, width is the width of a single channel, and hight is the height of a single channel, all of which belong to the natural number set scope; 2) The convolution kernel of the 2D convolution layer of the residual block in the regular convolution block is set to K_2, the size of the convolution kernel is 2k_size, and the stride is 2Stride. The first residual block performs a 2D convolution operation on the initial feature extraction result and connects the first batch processing normalization layer and the first ReLU activation function. The first ReLU activation function connects a 2D convolution operation layer and the second batch processing normalization layer; 3) The initial feature extraction result and the output of the second batch normalization layer are used as the input of the second residual block. The second residual block performs a 2D convolution operation on the input. The output of the 2D convolution operation is used as the input of the third batch normalization layer. The output of the third batch normalization layer is used as the input of the second ReLU activation function. The output of the second ReLU activation function is subjected to another 2D convolution operation, and the result is input into the fourth batch normalization layer. 4) The output of the previous regular convolution block is used as the input of the current regular convolution block. In the current regular convolution block, the first residual block performs a 2D convolution operation on the output of the previous regular convolution block, and connects the first batch processing normalization layer and the first ReLU activation function. The first ReLU activation function connects a 2D convolution operation layer and the second batch processing normalization layer; 5) The output of the previous regular convolution block and the output of the second batch normalization layer are used as the input of the second residual block. The second residual block performs a 2D convolution operation on the input. The output of the 2D convolution operation is used as the input of the third batch normalization layer. The output of the third batch normalization layer is used as the input of the second ReLU activation function. The output of the second ReLU activation function is subjected to another 2D convolution operation. The result is input into the fourth batch normalization layer. Repeat steps 4) and 5) to complete the operations of the third and fourth regular convolution blocks and obtain semantic information. Among them, C is the number of channels, M is the width of a single channel, and N is the height of a single channel, all of which belong to the natural number set range.
4. The remote sensing image data compression and transmission method according to claim 1, characterized in that: The important feature selection module includes a global average pooling layer, a first fully connected layer, a gated activation function, a Sigmoid activation function, and a mask matrix generator network that are connected in sequence.
5. The remote sensing image data compression and transmission method according to claim 1, characterized in that: The joint source-channel coding module includes a source encoder and a channel encoder; the source encoder includes four convolutional layers connected in sequence, each convolutional layer is followed by a ReLU activation function; the channel encoder includes Gaussian channel and Rayleigh fading channel coding; the Gaussian channel and Rayleigh fading channel coding add random Gaussian noise and randomly generated Rayleigh fading coefficients to the output of the source encoder to obtain an interfering coding result.
6. The remote sensing image data compression and transmission method according to claim 1, characterized in that: The decoder includes four deconvolution layers connected in sequence, the first three convolution layers are each followed by a ReLU activation function, and the last deconvolution layer is followed by a Sigmoid activation function.
7. The remote sensing image data compression and transmission method according to claim 1, characterized in that: The downstream task module includes an average pooling layer and a fully connected layer connected in sequence, the output of the fully connected layer is used as the input of the argmax function, and the output of the argmax function is the final output result.
8. The remote sensing image data compression and transmission method according to claim 1, characterized in that: For the trained remote sensing image data compression and transmission model, the semantic feature extraction module, important feature selection module, and joint source channel coding module are deployed at the remote sensing data acquisition end, and the decoder and downstream task modules are deployed at the ground end.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.
10. A computer program product comprising a computer program / instructions; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Joint information source channel coding method for image semantic communication
CN117879765A
Remote sensing image satellite-ground transmission method, terminal equipment and computer program product
CN118573868A
Cited By
Wireless image transmission method based on channel semantic importance
CN121842163A
