Remote sensing image transmission method and device based on source channel elastic coding, equipment and storage medium

Abstract

The invention discloses a remote sensing image transmission method and device based on source channel elastic coding, equipment and a storage medium, and the method comprises the steps: obtaining a target remote sensing image, and extracting a corresponding semantic feature from the target remote sensing image; determining a feature selection sequence based on the attention values corresponding to the semantic features; selecting corresponding key features from the semantic features based on a feature selection sequence; source-channel joint coding is carried out on the key features to obtain joint coding features, and the source-channel joint coding at least comprises source compression coding and enhancement training based on channel noise; and sending the joint coding features to a receiving end for decoding and classification. Through the above mode, the compression ratio and the downstream task performance are effectively improved by using key semantic feature selection, joint coding of multi-noise training and task-oriented end-to-end optimization, the method can adapt to a complex and changeable communication environment, and efficient transmission of remote sensing images is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a remote sensing image transmission method and device based on source channel elastic coding, equipment and storage medium. BACKGROUND

[0002] In recent years, with the progress of satellite imaging technology and the popularization of remote sensing instruments, the demand for remote sensing image compression and transmission is increasing. In complex communication environments such as weak communication links and poor wireless communication environments, simple data compression is not enough, and the most commonly used is joint source channel coding technology (JSCC). JSCC combines source and channel coding, which can improve the communication reliability in noise and interference environment. However, the performance of JSCC depends on the accuracy of the channel model. In dynamic environment, signal-to-noise ratio, noise type and signal fading change continuously. The model trained on fixed signal-to-noise ratio shows significant performance decline under these conditions, and the channel coding transmission accuracy is insufficient, and the dynamic adaptability to complex communication environment is insufficient, which is difficult to realize efficient and reliable communication in remote sensing image communication scenarios.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a remote sensing image transmission method, device, equipment and storage medium based on source channel elastic coding, which aims to solve the technical problem that the remote sensing image in the prior art is difficult to realize efficient transmission in complex communication environment.

[0005] To achieve the above purpose, the present application provides a remote sensing image transmission method based on source channel elastic coding, the method comprising:

[0006] Obtaining a target remote sensing image, extracting corresponding semantic features from the target remote sensing image;

[0007] Determining a feature selection sequence based on the attention value corresponding to the semantic features;

[0008] Selecting corresponding key features from the semantic features based on the feature selection sequence;

[0009] Jointly encoding the key features based on source and channel to obtain joint encoding features, wherein the joint encoding of source and channel at least includes source compression encoding and enhancement training based on channel noise;

[0010] Sending the joint encoding features to the receiving end for decoding and classification.

[0011] In an embodiment, the step of obtaining a target remote sensing image and extracting corresponding semantic features from the target remote sensing image comprises:

[0012] The obtained target remote sensing image is input into a residual unit for feature extraction to obtain embedding features, and the residual unit at least includes a plurality of residual blocks, and the residual block includes a first convolutional layer, a first normalization layer, a nonlinear activation layer, a second convolutional layer and a second normalization layer connected in sequence;

[0013] The embedding features are input into a state space unit for processing to obtain semantic features of the target remote sensing image.

[0014] In an embodiment, the step of inputting the embedding features into the state space unit for processing to obtain semantic features of the target remote sensing image comprises:

[0015] The embedding features are input into the state space unit, and the embedding features are subjected to layer normalization processing based on the state space unit to obtain normalized embedding features;

[0016] The normalized embedding features are subjected to scanning processing based on the state space unit to obtain initial direction features, and the direction of the scanning processing at least includes horizontal forward, horizontal backward, vertical forward and vertical backward;

[0017] The initial direction features are processed based on a selective state space model of the state space unit to obtain state space direction features;

[0018] The semantic features of the target remote sensing image are obtained based on the state space direction features.

[0019] In an embodiment, the step of determining a feature selection sequence based on the attention value corresponding to the semantic features comprises:

[0020] The semantic features are subjected to layer normalization processing to obtain normalized semantic features;

[0021] The normalized semantic features are processed based on a self-attention mechanism to obtain an attention value;

[0022] A feature selection sequence is generated based on the attention value and the selection quantity corresponding to the target signal-to-noise ratio.

[0023] In an embodiment, the channel noise is additive white Gaussian noise, and the step of source-channel joint encoding the key features to obtain joint encoding features comprises:

[0024] The key features are compressed and encoded based on a plurality of hollow convolutional layers to obtain compressed and encoded features;

[0025] Based on additive white Gaussian noise, the compression coding feature is enhanced and trained to obtain a noisy compression coding feature, which is used as a joint coding feature.

[0026] In one embodiment, the step of performing enhancement training on the compression coding feature based on additive white Gaussian noise to obtain the noisy compression coding feature includes:

[0027] Obtaining the corresponding relationship between additive white Gaussian noise, compression coding features, complex Gaussian random variables and noise-added compression coding features;

[0028] Obtain additive white Gaussian noise and complex Gaussian random variables;

[0029] Based on the compression coding feature, the additive white Gaussian noise, the complex Gaussian random variable and the corresponding relationship, a noise-added compression coding feature is obtained.

[0030] In one embodiment, the step of sending the joint coding feature to the receiving end for decoding and classification includes:

[0031] The joint coding features are sent to a receiving end so that the receiving end decodes the joint coding features to obtain reconstructed remote sensing image data, performs convolution processing and average pooling processing on the reconstructed remote sensing image data to obtain global features, and determines the classification result of the target remote sensing image based on the global features.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a remote sensing image transmission device based on source channel elastic coding, and the remote sensing image transmission device based on source channel elastic coding includes:

[0033] A feature extraction module is used to obtain a target remote sensing image and extract corresponding semantic features from the target remote sensing image;

[0034] A key feature elastic selection module, configured to determine a feature selection sequence based on the attention values ​​corresponding to the semantic features;

[0035] The key feature elastic selection module is further configured to select corresponding key features from the semantic features based on the feature selection sequence;

[0036] a source-channel joint coding module, configured to perform source-channel joint coding on the key features to obtain joint coding features, wherein the source-channel joint coding includes at least source compression coding and enhanced training based on channel noise;

[0037] The decoding and downstream task module is used to send the joint coding features to the receiving end for decoding and classification.

[0038] In addition, to achieve the above object, the present application also provides a remote sensing image transmission device based on source channel elastic coding, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the remote sensing image transmission method based on source channel elastic coding as described above.

[0039] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the remote sensing image transmission method based on source channel elastic coding as described above.

[0040] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the remote sensing image transmission method based on source channel elastic coding as described above.

[0041] The present application provides a remote sensing image transmission method based on source channel elastic coding, acquires a target remote sensing image, extracts corresponding semantic features from the target remote sensing image, determines a feature selection sequence based on attention values corresponding to the semantic features, selects corresponding key features from the semantic features based on the feature selection sequence, performs source channel joint coding on the key features to obtain joint coding features, and the source channel joint coding at least includes source compression coding and channel noise-based enhanced training, and sends the joint coding features to a receiving end for decoding and classification. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0044] Figure 1Flowchart of the remote sensing image transmission method based on source channel elastic coding of the first embodiment of the application;

[0045] Figure 2 Residual block structure diagram of the remote sensing image transmission method based on source channel elastic coding of the first embodiment of the application;

[0046] Figure 3 Key feature elastic selection module structure diagram of the remote sensing image transmission method based on source channel elastic coding of the first embodiment of the application;

[0047] Figure 4 Overall architecture diagram of the remote sensing image transmission method based on source channel elastic coding of the first embodiment of the application;

[0048] Figure 5 Flowchart of the remote sensing image transmission method based on source channel elastic coding of the second embodiment of the application;

[0049] Figure 6 Module structure diagram of the remote sensing image transmission device based on source channel elastic coding of the embodiment of the application;

[0050] Figure 7 Device structure diagram of the hardware running environment involved in the remote sensing image transmission method based on source channel elastic coding of the embodiment of the application.

[0051] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the application, and are not used to limit the application.

[0053] In order to better understand the technical solutions of the application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0054] The main solution of the embodiment of the application is: obtaining a target remote sensing image, extracting corresponding semantic features from the target remote sensing image; determining a feature selection sequence based on the attention value corresponding to the semantic features; selecting corresponding key features from the semantic features based on the feature selection sequence; performing source channel joint coding on the key features to obtain joint coding features, and the source channel joint coding at least includes source compression coding and enhancement training based on channel noise; transmitting the joint coding features to the receiving end for decoding and classification.

[0055] At present, in a dynamic environment, the signal-to-noise ratio, noise type and signal fading change continuously, the model trained on the fixed signal-to-noise ratio shows significant performance decline under these conditions, the channel coding transmission accuracy is insufficient, the dynamic adaptability to the complex communication environment is insufficient, and it is difficult to realize efficient and reliable communication in the remote sensing image communication scene.

[0056] The present application provides a solution, which effectively improves the compression rate and the performance of downstream tasks through attention-enhanced key semantic feature selection, joint encoding of multi-noise training and task-oriented end-to-end optimization, guarantees the balance between the compression rate and the classification accuracy of remote sensing images, can adapt to complex and changeable communication environment, improves the efficiency and robustness of remote sensing image transmission, realizes efficient transmission of remote sensing images, and solves the technical problem that remote sensing images are difficult to realize efficient transmission in complex communication environment.

[0057] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a remote sensing image transmission device based on source channel elastic coding, etc., and the present embodiment does not make specific limitation thereon. In the following, the remote sensing image transmission device based on source channel elastic coding is taken as an example to describe the present embodiment and each of the following embodiments.

[0058] The present application embodiment provides a remote sensing image transmission method based on source channel elastic coding, referring to Figure 1 , Figure 1 The present application provides a remote sensing image transmission method based on source channel elastic coding, referring to

[0059] In the present embodiment, the remote sensing image transmission method based on source channel elastic coding comprises steps S10-S50:

[0060] Step S10, acquiring a target remote sensing image, and extracting corresponding semantic features from the target remote sensing image;

[0061] It should be noted that the target remote sensing image is the remote sensing image that needs to be transmitted, that is, the remote sensing image transmitted from the sending end to the receiving end. Generally, the sending end is the star end (satellite end), and the receiving end is the ground end (ground end).

[0062] In addition, it should be noted that the present embodiment uses a feature extraction module to extract rich semantic features from the target remote sensing image. The feature extraction module comprises a connected residual unit and a state space (Mamba) unit.

[0063] In a feasible implementation manner, step S10 can comprise steps S101-S102:

[0064] Step S101, input the obtained target remote sensing image into a residual unit for feature extraction to obtain embedding features;

[0065] It should be noted that the residual unit includes at least a plurality of residual blocks, for reference Figure 2 The residual block includes a first convolutional layer, a first normalization layer, a nonlinear activation layer, a second convolutional layer and a second normalization layer connected in sequence, and the residual blocks are connected by a residual connection. In this embodiment, the residual unit is provided with two residual blocks, and the nonlinear activation layer in the residual block usually uses a ReLU (Linear rectification function, rectified linear unit) layer, and the first normalization layer and the second normalization layer both adopt a batch normalization processing mode.

[0066] It can be understood that for an input remote sensing image X with a channel number of C0, a width of W0 and a height of H0, the residual unit composed of two residual blocks is used for feature extraction to obtain embedding features, and the calculation relationship is as follows:

[0067] F = A(X, θ f ), F ∈ R H×W×C

[0068] In the formula, F represents the embedding features, X represents the target remote sensing image, A(·) represents the residual unit, θ f represents the key parameters of the residual unit, C represents the channel number of the embedding features, W represents the width of the embedding features, and H represents the height of the embedding features.

[0069] Step S102, input the embedding features into a state space unit for processing to obtain semantic features of the target remote sensing image.

[0070] It should be noted that the state space unit includes at least a plurality of Mamba blocks, and in this embodiment, the state space unit is provided with two Mamba blocks.

[0071] In a possible implementation, step S102 can include: inputting the embedding features into the state space unit, performing layer normalization processing on the embedding features based on the state space unit to obtain normalized embedding features; performing scanning processing on the normalized embedding features based on the state space unit to obtain initial direction features, the direction of the scanning processing including at least horizontal forward, horizontal backward, vertical forward and vertical backward; performing processing on the initial direction features based on a selective state space model of the state space unit to obtain state space direction features; and obtaining semantic features of the target remote sensing image based on the state space direction features.

[0072] It should be noted that the preset direction is a direction used in a preset scanning process, and at least includes a horizontal forward direction, a horizontal backward direction, a vertical forward direction, and a vertical backward direction.

[0073] It can be understood that after the embedding feature F is input into the Mamba unit, layer normalization processing is first performed to obtain a normalized embedding feature, i.e., a normalized embedding feature F'. The normalized embedding feature F' is subjected to scanning processing in four directions (a horizontal forward direction, a horizontal backward direction, a vertical forward direction, and a vertical backward direction) to obtain four initial direction features, i.e., initial direction features, which can be respectively represented as a horizontal forward embedding feature F hf ∈R HW×C , a horizontal backward embedding feature F hb ∈R HW×C , a vertical forward embedding feature F vf ∈R HW×C , and a vertical backward embedding feature F vb ∈R HW×C . After being subjected to selective state space model and linear layer processing in the Mamba block, corresponding Mamba features, i.e., state space direction features, are obtained, and a calculation relationship is as follows:

[0074] Y hf = S6(Linear(F hf ))

[0075] Y hb = Y6(Linear(F hb ))

[0076] Y vf = S6(Linear(F vf ))

[0077] Y vb = Y6(Linear(F vb ))

[0078] In the formula, Y hf represents a horizontal forward state space direction feature, Y hb represents a horizontal backward state space direction feature, Y vf represents a vertical forward state space direction feature, Y vb represents a vertical backward state space direction feature, S6(·) represents a selective state space model in the Mamba block, and Linear(·) represents a linear layer. The output of the Mamba block after scanning in four directions can be represented as:

[0079] Y = Y hf + Y hb + Y vf + Y vb

[0080] Where Y represents the output of the Mamba block, Y hf Represents the horizontal forward state space direction feature, Y hb Represents the horizontal backward state space direction feature, Y vf Represents the vertical forward state space direction feature, Y vb Represents the vertical backward state space direction feature.

[0081] It should be understood that since two Mamba blocks are set in this embodiment, the output of the first Mamba block needs to be input into the second Mamba block for processing, and the processing process of the two Mamba blocks is similar. The semantic features output after the two Mamba blocks can be expressed as:

[0082] S=Mamba(F,θ m )

[0083] In the formula, S represents semantic features, F represents embedding features, and θ m Represents the trainable parameters of the Mamba block.

[0084] Step S20, determining a feature selection sequence based on the attention value corresponding to the semantic feature;

[0085] It should be noted that because traditional solutions uniformly compress all features, which can easily lead to redundant information occupying bandwidth, this embodiment uses an attention mechanism to select task-related channels to achieve task-driven dynamic feature compression, reducing bandwidth costs while maintaining task performance. In specific implementation, this embodiment uses a key feature elastic selection module to extract key semantic features, namely key features.

[0086] In addition, it should be noted that, refer to Figure 3 The key feature elastic selection module includes a layer normalization layer (LayerNorm), a self-attention layer (self attention), a nonlinear activation layer (ReLU), a linear layer (Linear), an activation layer (softmax) and a signal-to-noise ratio selection layer (SNR selection layer) connected in sequence, where the SNR selection layer can be implemented through three fully connected layers.

[0087] In a feasible implementation, step S20 may include: performing layer normalization processing on the semantic features to obtain normalized semantic features; processing the normalized semantic features based on the self-attention mechanism to obtain an attention value; and generating a feature selection sequence based on the attention value and the selection quantity corresponding to the target signal-to-noise ratio.

[0088] It can be understood that after the semantic feature is input, it is first normalized by the LayerNorm layer to obtain the normalized semantic feature, that is, the normalized semantic feature. Then, the corresponding attention value is calculated using the self-attention mechanism. After passing through the ReLU nonlinear activation layer, Linear linear layer, Softmax activation layer and SNR selection layer, the importance Mask value is generated. The sequence of importance Mask values ​​is the feature selection sequence.

[0089] It should be understood that the target signal-to-noise ratio, i.e., the signal-to-noise ratio (SNR) input to the SNR selection layer, can be set according to actual conditions. In this embodiment, the size of the target signal-to-noise ratio determines the number of key features to be extracted, that is, the size of the target signal-to-noise ratio determines the number of selections. According to the attention value and the number of selections corresponding to the target signal-to-noise ratio, an importance mask value is generated. A value of 1 represents selection, and a value of 0 represents non-selection, thereby forming the final feature selection sequence.

[0090] Step S30, selecting corresponding key features from the semantic features based on the feature selection sequence;

[0091] It should be noted that the key features are obtained by multiplying the mask value in the feature selection sequence with the semantic feature. The overall processing process of the key feature elastic selection module can be expressed by the following calculation relationship:

[0092] S′=KFESM(S;θ k ),S′∈R M×H×W

[0093] In the formula, S′ represents the key feature, S represents the semantic feature, KFESM(·) represents the key feature elastic selection module, θ k Represents the trainable parameters of the key feature elastic selection module, M represents the number of channels of the key feature, M<<C, and C is the number of channels of the embedded feature.

[0094] It can be understood that through the feature extraction module and the key feature elastic selection module, redundant information can be effectively removed and the amount of transmitted data can be reduced, which can achieve an extremely high compression ratio while maintaining high downstream task performance.

[0095] Step S40, performing source-channel joint coding on the key features to obtain joint coding features, wherein the source-channel joint coding at least includes source compression coding and enhanced training based on channel noise;

[0096] It should be noted that the source-channel joint coding in this embodiment includes at least source compression coding and channel noise-based enhancement training. By combining source and channel characteristics, compression coding of remote sensing images is achieved, enabling the transmitted data to adapt to complex channel variations. In specific implementations, a source-channel joint coding module is used to perform source-channel joint coding on key features. The output of this module is the jointly coded feature.

[0097] It can be understood that the compression encoding of key features can be achieved through the dilated convolution operation of multiple dilated convolution layers.

[0098] It should be understood that this embodiment enhances the robustness of remote sensing image transmission in a complex wireless communication environment, can cope with noise of different types and intensities and channel changes, and ensure the reliability of data transmission.

[0099] Step S50: sending the joint coding feature to a receiving end for decoding and classification.

[0100] It should be noted that this embodiment decodes the transmitted joint coding features through the decoding and downstream task module and realizes the target classification task at the receiving end. The decoding and downstream task module includes a decoding module and a downstream task module. Figure 4 The decoding module and downstream task module are usually set at the receiving end (ground end), the feature extraction module, the key feature elastic selection module, and the source-channel joint coding module are usually set at the transmitting end (satellite end), and data is transmitted between the source-channel joint coding module and the decoding module through a wireless channel.

[0101] In a feasible embodiment, step S50 may include: sending the joint coding features to a receiving end so that the receiving end decodes the joint coding features to obtain reconstructed remote sensing image data, performing convolution processing and average pooling processing on the reconstructed remote sensing image data to obtain global features, and determining the classification results of the target remote sensing image based on the global features.

[0102] It should be noted that the decoding module is used to decode the compressed channel information (i.e., the joint coding features) to obtain the reconstructed remote sensing image, i.e., the reconstructed remote sensing image data. The decoding process usually corresponds to the encoding process. If the compression encoding uses five dilated convolution layers for dilated convolution operations, then the decoding process uses five dilated deconvolution layers for dilated deconvolution operations. The calculation relationship is as follows:

[0103]

[0104] Where, represents the reconstructed remote sensing image data, D' represents the joint encoded features, DeConvs(·) represents the deconvolution operation of the five dilated convolution layers, and θ d represents the network parameters of the dilated deconvolution layer.

[0105] It can be understood that the downstream task module is used to complete the classification task of the remote sensing image, and the global features are extracted from the reconstructed remote sensing image data through the convolution operation of the standard convolution layer and the average pooling operation of the average pooling layer, and the calculation relationship is as follows:

[0106]

[0107] In the formula, represents the reconstructed remote sensing image data, q represents the global features, Pooling(·) represents the average pooling operation, and Conv(·) represents the convolution operation. Then the classification task is completed through the full connection layer and the Softmax layer to obtain the final classification result. The classification result is usually a predicted probability value, and the calculation relationship is as follows:

[0108]

[0109] In the formula, represents the classification result, q represents the global features, FC(·) represents the full connection layer, and softmax(·) represents the Softmax layer.

[0110] It should be understood that the embodiment is task-oriented for optimization, so that the transmitted data is more conducive to the completion of the downstream task, and the accuracy and efficiency of the task completion are improved.

[0111] Further, the cross-entropy loss is calculated according to the classification result, and the parameters of all modules are updated based on the cross-entropy loss. The calculation relationship of the cross-entropy loss is as follows:

[0112]

[0113] In the formula, L represents the cross-entropy loss, y k represents the true value of the kth class, represents the predicted result (classification result) of the kth class, and K represents the number of categories.

[0114] In a specific implementation, the feature extraction module, the key feature elastic selection module, the source channel joint coding module, and the decoding and downstream task module can be integrated into a remote sensing image compression classification model. First, the feature extraction module, the key feature elastic selection module, the source channel joint coding module, and the decoding and downstream task module are built, and the parameters required in the initialization module are initialized, including the convolution layer parameters, the initial weights and biases of each module, etc., for example: all the convolution kernel sizes are set to 3*3; the channel numbers of the convolution layers in the two residual units in the feature extraction module are set to 64, 128, 256, and 512; the size of the matrix in the state selection space S6 is set to 256*128; the channel numbers of the five hollow convolution layers in the source channel joint coding module are set to 256, 128, 64, 32, and 16. Next, the remote sensing image data required for model training is prepared, and the data is divided into a training set, a validation set, and a test set according to the training and testing requirements, for example: the ratio of training / validation / test images is set to 5:1:4. Then, in the model training stage, the training parameters need to be determined first, including the training rounds, the optimizer, the learning rate, the batch size, etc., for example: using Adam as the optimizer, the batch size is set to 64, the initial learning rate is set to 0.0001, the maximum training rounds are set to 60, the training data prepared in advance is input into the extraction module, and then the key feature elastic selection module, the source channel joint coding module, and the decoding and downstream task module are sequentially passed to obtain the forward propagation calculation result, and the loss function is calculated. According to the loss function, back propagation is performed, and the network parameters are updated for multiple rounds of training. The trained model is saved for subsequent deployment. Finally, the trained model is deployed on the sending end and the receiving end. The decoding and downstream task module is usually deployed on the receiving end (ground end), and the feature extraction module, the key feature elastic selection module, and the source channel joint coding module are usually deployed on the sending end (star end). For the remote sensing images that need to be transmitted, the model deployed on the sending end is used for compression coding, and after transmission through the wireless channel, decoding is performed on the receiving end to complete the downstream task, realizing efficient compression and accurate transmission of remote sensing images.

[0115] The embodiment provides a remote sensing image transmission method based on source channel elastic coding. A target remote sensing image is acquired, and corresponding semantic features are extracted from the target remote sensing image. A feature selection sequence is determined based on attention values corresponding to the semantic features. Corresponding key features are selected from the semantic features based on the feature selection sequence. The key features are jointly coded by source channel coding to obtain joint coding features. The source channel joint coding at least includes source compression coding and enhanced training based on channel noise. The joint coding features are sent to a receiving end for decoding and classification. The embodiment effectively improves the compression rate and the performance of a downstream task by key semantic feature selection based on attention enhancement, joint coding based on multi-noise training, and task-oriented end-to-end optimization, balances the compression rate and classification accuracy of the remote sensing image, is suitable for complex and changeable communication environments, improves the efficiency and robustness of remote sensing image transmission, and realizes efficient remote sensing image transmission.

[0116] Based on the first embodiment of the application, the same or similar contents as the above-mentioned embodiment one can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 5 , and the step S40 can include steps S401-S402:

[0117] In step S401, the key features are compressed and coded based on a plurality of dilated convolution layers to obtain compressed coding features.

[0118] It should be noted that the dilated convolution operation of the dilated convolution layer is used to realize the compression coding of the key features, and five dilated convolution layers are set in the specific implementation.

[0119] It can be understood that the compression coding of the key features is realized by the five dilated convolution layers to obtain the latent representation, i.e., the compressed coding features, and the calculation relationship is as follows:

[0120] D = DilatedConvs (S'; θ e )

[0121] In the formula, D represents the compressed coding features, DilatedConvs(·) represents the dilated convolution operation of the five dilated convolution layers, and θ e represents the network parameters of the dilated convolution layer.

[0122] In step S402, the compressed coding features are enhanced and trained based on additive white Gaussian noise to obtain noise-added compressed coding features, and the noise-added compressed coding features are used as joint coding features.

[0123] It should be noted that after completing the compression coding of the source, channel noise is further introduced for enhanced training to obtain the noisy features, that is, the noisy compressed coding features. The noisy compressed coding features at this time are the final joint coding features.

[0124] In a feasible implementation, the steps of performing enhanced training on the compression coding features based on additive white Gaussian noise to obtain the noisy compression coding features may include: obtaining the correspondence between the additive white Gaussian noise, the compression coding features, the complex Gaussian random variables and the noisy compression coding features; obtaining the additive white Gaussian noise and the complex Gaussian random variables; and obtaining the noisy compression coding features based on the compression coding features, the additive white Gaussian noise, the complex Gaussian random variables and the correspondence.

[0125] It should be noted that the channel noise used in this embodiment is additive white Gaussian noise. The corresponding relationship between additive white Gaussian noise, compression coding features, complex Gaussian random variables, and the noise-added compression coding features, i.e., the calculation relationship of the noise-added compression coding features, is as follows:

[0126] D′=D+Noise aWGN +R·D+E

[0127] In the formula, D′ represents the noise compression coding feature (joint coding feature), D represents the compression coding feature, Noise AWGN represents additive white Gaussian noise, R and E are complex Gaussian random variables, and represent the Rayleigh fading of the channel. By introducing Gaussian white noise and Rayleigh fading for joint channel training, the robustness of the model in complex channel conditions is further improved.

[0128] In a specific implementation, it is assumed that all convolution kernel sizes are 3x3, in the source channel joint coding module, the channel numbers of the five hollow convolution layers are 256, 128, 64, 32, and 16 respectively, the strides of the first two layers are 2, the strides of the remaining layers are 1, the target remote sensing image to be transmitted is a tensor with a size of 16x4x4, and the compression ratio is 768. Under the compression ratio of 768, the classification accuracy of the embodiment scheme in the Gaussian white noise channel can reach 83.27%, and the classification accuracy in the Rayleigh fading channel can reach 82.19%. The classification accuracy of the AlexNet (deep convolutional neural network) model in the Gaussian white noise channel is 35.18%, and the classification accuracy in the Rayleigh fading channel is 42.19%. The classification accuracy of the GoogleNet (deep learning convolutional neural network architecture) model in the Gaussian white noise channel is 63.81%, and the classification accuracy in the Rayleigh fading channel is 75.12%. The classification accuracy of the DenseNet (dense connection convolutional network) model in the Gaussian white noise channel is 75.52%, and the classification accuracy in the Rayleigh fading channel is 76.91%. The classification accuracy of the ADJSCC (attention-based joint source channel coding) model in the Gaussian white noise channel is 45.41%, and the classification accuracy in the Rayleigh fading channel is 50.94%. It can be seen that, compared with other traditional schemes, the embodiment can effectively extract and utilize key information, is less affected by channel noise interference, and can realize efficient information compression and accurate classification on remote sensing images.

[0129] The embodiment provides a remote sensing image transmission method based on source channel elastic coding. Key features are compressed and encoded based on a plurality of hollow convolution layers to obtain compressed and encoded features. The compressed and encoded features are enhanced and trained based on additive Gaussian white noise to obtain noisy compressed and encoded features, and the noisy compressed and encoded features are used as joint coding features. The embodiment effectively improves the compression rate and the performance of downstream tasks through attention-enhanced key semantic feature selection, multi-noise training joint coding, and task-oriented end-to-end optimization, balances the compression rate and classification accuracy of remote sensing images, adapts to complex and changeable communication environments, improves the efficiency and robustness of remote sensing image transmission, and realizes efficient remote sensing image transmission.

[0130] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the remote sensing image transmission method based on source channel elastic coding of the present application. More forms of simple transformation based on the technical concept are within the protection scope of the present application.

[0131] The present application also provides a remote sensing image transmission device based on source channel elastic coding, which is described with reference to Figure 6 The remote sensing image transmission device based on source channel elastic coding comprises:

[0132] The feature extraction module 10 is configured to obtain a target remote sensing image and extract corresponding semantic features from the target remote sensing image.

[0133] The key feature elastic selection module 20 is configured to determine a feature selection sequence based on the attention values corresponding to the semantic features.

[0134] The key feature elastic selection module 20 is further configured to select corresponding key features from the semantic features based on the feature selection sequence.

[0135] The source channel joint coding module 30 is configured to perform source channel joint coding on the key features to obtain joint coding features, and the source channel joint coding at least includes source compression coding and enhanced training based on channel noise.

[0136] The decoding and downstream task module 40 is configured to send the joint coding features to a receiving end for decoding and classification.

[0137] In a feasible implementation, the feature extraction module 10 is further configured to input the obtained target remote sensing image into a residual unit for feature extraction to obtain embedded features, and the residual unit at least includes a plurality of residual blocks, and each residual block includes a first convolutional layer, a first normalization layer, a nonlinear activation layer, a second convolutional layer and a second normalization layer connected in sequence.

[0138] The embedded features are input into a state space unit for processing to obtain semantic features of the target remote sensing image.

[0139] In a feasible implementation, the feature extraction module 10 is further configured to input the embedded features into a state space unit, perform layer normalization processing on the embedded features based on the state space unit to obtain normalized embedded features.

[0140] Perform scanning processing on the normalized embedded features based on the state space unit to obtain initial direction features, and the direction of the scanning processing at least includes horizontal forward, horizontal backward, vertical forward and vertical backward.

[0141] Perform processing on the initial direction features based on a selective state space model of the state space unit to obtain state space direction features.

[0142] Obtain semantic features of the target remote sensing image based on the state space direction features.

[0143] In a feasible implementation, the key feature elastic selection module 20 is further configured to perform layer normalization processing on the semantic features to obtain normalized semantic features.

[0144] The normalized semantic features are processed based on a self-attention mechanism to obtain attention values;

[0145] Based on the attention values and a selected number corresponding to a target signal-to-noise ratio, a feature selection sequence is generated.

[0146] In a feasible implementation, the source channel joint encoding module 30 is further configured to compress and encode the key features based on multiple hollow convolutional layers to obtain compressed and encoded features.

[0147] The compressed and encoded features are enhanced and trained based on additive white Gaussian noise to obtain noise-added compressed and encoded features, and the noise-added compressed and encoded features are taken as joint encoding features.

[0148] In a feasible implementation, the source channel joint encoding module 30 is further configured to obtain a corresponding relationship among additive white Gaussian noise, compressed and encoded features, complex Gaussian random variables, and noise-added compressed and encoded features.

[0149] The additive white Gaussian noise and the complex Gaussian random variables are obtained.

[0150] Based on the compressed and encoded features, the additive white Gaussian noise, the complex Gaussian random variables, and the corresponding relationship, the noise-added compressed and encoded features are obtained.

[0151] In a feasible implementation, the decoding and downstream task module 40 is further configured to send the joint encoding features to a receiving end, so that the receiving end decodes the joint encoding features to obtain reconstructed remote sensing image data, performs convolution processing and average pooling processing on the reconstructed remote sensing image data to obtain global features, and determines a classification result of the target remote sensing image based on the global features.

[0152] The remote sensing image transmission device based on source channel elastic encoding provided in the application adopts the remote sensing image transmission method based on source channel elastic encoding in the above embodiments, and can solve the technical problem that remote sensing images are difficult to implement efficient transmission in a complex communication environment. Compared with the prior art, the remote sensing image transmission device based on source channel elastic encoding provided in the application has the same beneficial effects as the remote sensing image transmission method based on source channel elastic encoding provided in the above embodiments, and other technical features in the remote sensing image transmission device based on source channel elastic encoding are the same as the features disclosed in the above method, which will not be repeated here.

[0153] The application provides a remote sensing image transmission device based on source channel elastic coding. The remote sensing image transmission device based on source channel elastic coding comprises at least one processor and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the remote sensing image transmission method based on source channel elastic coding in the above embodiment one.

[0154] Reference will be made to the following Figure 7 which shows a structural diagram of the remote sensing image transmission device based on source channel elastic coding suitable for implementing the embodiments of the application. The remote sensing image transmission device based on source channel elastic coding in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 7 The remote sensing image transmission device based on source channel elastic coding shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the application.

[0155] As Figure 7As shown, the remote sensing image transmission device based on source channel resilient coding can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a ROM (Read Only Memory) 1002 or programs loaded from a storage apparatus 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for operation of the remote sensing image transmission device based on source channel resilient coding are also stored. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input apparatuses 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the remote sensing image transmission device based on source channel resilient coding to perform wireless or wired communication with other devices to exchange data. Although the remote sensing image transmission device based on source channel resilient coding with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0156] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carrying computer program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0157] The remote sensing image transmission device based on source channel elastic coding provided in the application adopts the remote sensing image transmission method based on source channel elastic coding in the above embodiment, and can solve the technical problem that remote sensing images are difficult to realize efficient transmission in a complex communication environment. Compared with the prior art, the remote sensing image transmission device based on source channel elastic coding provided in the application has the same beneficial effects as the remote sensing image transmission method based on source channel elastic coding provided in the above embodiment, and other technical features in the remote sensing image transmission device based on source channel elastic coding are the same as the features disclosed in the above embodiment method, and will not be repeated here.

[0158] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0159] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0160] The present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the remote sensing image transmission method based on source channel elastic coding in the above embodiment.

[0161] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0162] The computer readable storage medium described above can be included in a remote sensing image transmission device based on source channel elastic coding, or can exist separately without being assembled into the remote sensing image transmission device based on source channel elastic coding.

[0163] The computer readable storage medium described above carries one or more programs, which, when executed by the remote sensing image transmission device based on source channel elastic coding, cause the remote sensing image transmission device based on source channel elastic coding to: acquire a target remote sensing image, extract corresponding semantic features from the target remote sensing image; determine a feature selection sequence based on the attention value corresponding to the semantic features; select corresponding key features from the semantic features based on the feature selection sequence; source channel joint coding is performed on the key features to obtain joint coding features, and the source channel joint coding at least includes source compression coding and channel noise based enhancement training; and the joint coding features are sent to the receiving end for decoding and classification.

[0164] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0165] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0166] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0167] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the remote sensing image transmission method based on source channel elastic coding described above, and can solve the technical problem that remote sensing images are difficult to implement efficient transmission in complex communication environments. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the remote sensing image transmission method based on source channel elastic coding provided by the above embodiments, and will not be described here.

[0168] The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the remote sensing image transmission method based on source channel elastic coding as described above.

[0169] The computer program product provided by the application can solve the technical problem that remote sensing images are difficult to be efficiently transmitted in a complex communication environment. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the remote sensing image transmission method based on source channel elastic coding provided by the above-mentioned embodiments, and are not described here.

[0170] The above is only some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A remote sensing image transmission method based on source channel elastic coding, characterized in that: The method comprises: Acquiring a target remote sensing image, and extracting corresponding semantic features from the target remote sensing image; Determining a feature selection sequence based on the attention value corresponding to the semantic feature; Based on the feature selection sequence, selecting corresponding key features from the semantic features; Performing source-channel joint coding on the key features to obtain joint coding features, wherein the source-channel joint coding includes at least source compression coding and enhanced training based on channel noise; The joint coding features are sent to a receiving end for decoding and classification.

2. The method according to claim 1, wherein The step of acquiring a target remote sensing image and extracting corresponding semantic features from the target remote sensing image comprises: Inputting the acquired target remote sensing image into a residual unit for feature extraction to obtain embedded features, wherein the residual unit includes at least a plurality of residual blocks, and the residual block includes a first convolutional layer, a first normalization layer, a nonlinear activation layer, a second convolutional layer, and a second normalization layer connected in sequence; The embedded features are input into a state space unit for processing to obtain semantic features of the target remote sensing image.

3. The method according to claim 2, wherein The step of inputting the embedded features into a state space unit for processing to obtain the semantic features of the target remote sensing image comprises: Inputting the embedded features into a state space unit, and performing layer normalization processing on the embedded features based on the state space unit to obtain normalized embedded features; Scanning the normalized embedded features based on the state space unit to obtain initial directional features, where the directions of the scanning processing include at least horizontal forward, horizontal backward, vertical forward, and vertical backward; processing the initial directional features based on a selective state space model of the state space unit to obtain a state space directional feature; Based on the state space directional features, semantic features of the target remote sensing image are obtained.

4. The method according to claim 1, wherein The step of determining a feature selection sequence based on the attention value corresponding to the semantic feature includes: Performing layer normalization processing on the semantic features to obtain normalized semantic features; Processing the normalized semantic features based on a self-attention mechanism to obtain an attention value; Based on the attention value and the number of selections corresponding to the target signal-to-noise ratio, a feature selection sequence is generated.

5. The method according to claim 1, wherein The channel noise is additive white Gaussian noise, and the step of performing source-channel joint coding on the key feature to obtain a joint coding feature includes: Based on multiple hole convolutional layers, the key features are compressed and encoded to obtain compressed coding features; Based on additive white Gaussian noise, the compression coding feature is enhanced and trained to obtain a noisy compression coding feature, which is used as a joint coding feature.

6. The method according to claim 5, wherein The step of performing enhanced training on the compression coding feature based on additive white Gaussian noise to obtain the noise-added compression coding feature comprises: Obtaining the corresponding relationship between additive white Gaussian noise, compression coding features, complex Gaussian random variables and noise-added compression coding features; Obtain additive white Gaussian noise and complex Gaussian random variables; Based on the compression coding feature, the additive white Gaussian noise, the complex Gaussian random variable and the corresponding relationship, a noise-added compression coding feature is obtained.

7. The method according to claim 1, wherein The step of sending the joint coding feature to the receiving end for decoding and classification includes: The joint coding features are sent to a receiving end so that the receiving end decodes the joint coding features to obtain reconstructed remote sensing image data, performs convolution processing and average pooling processing on the reconstructed remote sensing image data to obtain global features, and determines the classification result of the target remote sensing image based on the global features.

8. A remote sensing image transmission device based on source channel elastic coding, characterized in that: The device comprises: A feature extraction module is used to obtain a target remote sensing image and extract corresponding semantic features from the target remote sensing image; A key feature elastic selection module, configured to determine a feature selection sequence based on the attention values ​​corresponding to the semantic features; The key feature elastic selection module is further configured to select corresponding key features from the semantic features based on the feature selection sequence; a source-channel joint coding module, configured to perform source-channel joint coding on the key features to obtain joint coding features, wherein the source-channel joint coding includes at least source compression coding and enhanced training based on channel noise; The decoding and downstream task module is used to send the joint coding features to the receiving end for decoding and classification.

9. A remote sensing image transmission device based on source channel elastic coding, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the remote sensing image transmission method based on source channel elastic coding as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the remote sensing image transmission method based on source channel elastic coding as described in any one of claims 1 to 7 are implemented.

Images