A loss-resistant image coding method for unstable satellite networks

Abstract

The present invention provides a loss-resistant image coding method for unstable satellite networks, comprising the following steps: step 1, establishing an end-to-end image coding and decoding network; step 2, during coding, spatially rearranging the potential features extracted by the encoder from the image, and inputting the rearranged potential features into a super-prior encoder for distribution modeling; step 3, inputting the combined features into a decoder for decoding to obtain a reconstructed image; step 4, in the image coding and decoding network training process, adopting a variety of double-tail drop strategies, channel random masking strategies and introducing a Gilbert-Elliott packet loss model; step 5, grouping and sending the compressed super-prior and potential features. The method of the present invention has good versatility in actual network evaluation and can prevent pixel errors or color deviations in the progressive transmission process. In addition, the method of the present invention introduces an acceptable computational complexity, making it suitable for practical applications.

Description

Technical Field

[0001] The invention relates to the field of image coding, and in particular to a loss-resistant image coding method for unstable satellite networks. Background Art

[0002] Compared with Internet-based or low-Earth orbit satellite transmission, geostationary orbit satellite communication has obvious advantages, including wide signal coverage and all-weather synchronization. These advantages make it particularly suitable for navigation, remote area communications and short burst data services because it is not easily affected by harsh environments. However, geostationary orbit satellite communications often have problems such as extremely low available bandwidth, large data packet transmission intervals, and severe packet loss, which poses a huge challenge to emergency image transmission.

[0003] Over the past few decades, image compression techniques have focused on reducing bitrate consumption under bandwidth constraints. From traditional rule-based image coding standards such as JPEG, BPG, and WebP to recent learned image compression methods, substantial progress has been made in enhancing compression performance. Given the high network latency, progressive coding, also known as scalable coding techniques, has been developed to allow image preview using received packets. These algorithms encode images into a single scalable bitstream, improving the quality of the reconstructed image as more complete packets are received. However, few researchers have considered the impact of packet loss on the reconstruction results, especially in learning-based methods.

[0004] Considering the encoder-decoder pair commonly deployed in transmission systems, the encoder attempts to remove redundancy in the input image and create a compact representation that encapsulates the essential information for reconstruction at the receiver. This information is distributed across various packets without any expected redundancy or protection mechanism. When packet loss occurs, the stored information is permanently lost and cannot be recovered during the decoding process. When the decoder receives incomplete data, the decoder faces reconstruction difficulties because the lost packets cause changes in the feature distribution that the decoder cannot adapt to. This unpredictable packet loss not only leads to a degradation in reconstruction quality, but also causes severe decoding errors such as pixel errors or color deviations. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for anti-loss image coding for unstable satellite networks in view of the deficiencies of the prior art, comprising the following steps:

[0006] Step 1, establishing an end-to-end image coding and decoding network, the image coding and decoding network includes a coding end and a decoding end, the coding end includes an encoder and a super-prior encoder, the decoding end includes a decoder and a super-prior decoder; a given input image is passed through the encoder to obtain compact potential features, the potential features are input into the super-prior encoder to obtain super-prior, the potential features and super-prior after entropy coding and quantization are binary code streams, after entropy decoding the binary code stream, the quantized super-prior is first input into the super-prior decoder, and the obtained output and the potential features are input into the decoder to complete decoding;

[0007] Step 2: During encoding, the latent features extracted by the encoder from the image are spatially rearranged, and the rearranged latent features are input into the super prior encoder for distribution modeling;

[0008] Step 3: The decoder uses the mask to mark the potential features, uses a convolutional neural network to aggregate the potential features and the mask to obtain the combined features, and inputs the combined features into the decoder for decoding to obtain the reconstructed image;

[0009] Step 4: During the image encoding and decoding network training process, multiple double-tail drop strategies, channel random masking strategies and the Gilbert-Elliott packet loss model are adopted;

[0010] Step 5: Pack and send the compressed hyper-prior and latent features in groups. When grouping, it is additionally determined whether the current channel sequence number is a multiple of 4. If it is, the current channel is divided into the next package to be sent. If not, no additional operation is performed.

[0011] Step 2 includes: the encoder network is committed to eliminating redundancy in the input image and has a compact representation in the spatial channel dimension. Different channels of the latent features independently contain information such as brightness, color or edges, which may be unrelated to each other. In the case of channel-by-channel transmission used in the present invention, if packet loss occurs in a channel that is unrelated to other channels, it may cause irreversible degradation of the reconstruction result. Therefore, from the perspective of the encoder, it is crucial that each channel has a related corresponding channel for error control. When encoding, the latent features are Rearrange the spatial channels and transform the potential features Divide into small cubes with a length of 2, a width of 2 and 4 channels. In each cube, put the data in the upper left corner of the cube into the first channel of the original cube in the order from left to right and from top to bottom. Similarly, the data in the upper right corner, lower left corner and lower right corner are rearranged to the second, third and fourth channels of the original cube respectively. After all the small cubes are rearranged, the rearranged potential features are obtained through quantification. . This process distributes the features of each original channel to 4 new channels, thereby enhancing the correlation between channels. When there is no packet loss, rearranging it back again on the decoder side does not change the input features of the decoder. If any channel is lost, its information can still be reconstructed using the remaining 3 channels, effectively minimizing unrecoverable errors.

[0012] In step 3, use a binary mask To annotate the rearranged latent features , where C=192 is the number of channels of potential features, is the mask of the rearranged latent features; if the mask of the ith rearranged latent feature Indicates that the i-th rearranged latent feature has been received ,if Indicates packet loss, the i-th rearranged latent feature is not received , , the received incomplete features are derived through the following formula:

[0013] ,

[0014] where ⊙ represents element-wise multiplication, represents the residual rearranged latent features received by the decoder after packet loss;

[0015] After the spatial channels are rearranged, the rearranged potential features are obtained The original arrangement of and the corresponding loss mask .

[0016] In step 3, two convolutional layers are deployed, each followed by a GELU (Gaussian Error Linear Unit) activation function to mask the loss. Modeling, aggregating incomplete features conditioned on packet loss, masking the loss and the rearranged potential features The concatenation is used to obtain the combined features, and then a convolutional layer and a GELU activation function are used for feature fusion, and the fused features are input into the decoder. By explicitly modeling the mask, the decoder can aggregate features according to the specific packet loss situation, thereby more effectively adapting to the changes in feature distribution caused by packet loss; at this time, the loss function L is:

[0017] ,

[0018] in Represents the quantized rearranged latent features The code rate, Represents the quantized hyper-prior The code rate, Represents a mask conditional aggregation operation, Represents a decoder, represents the reconstruction loss between the input image x and the output of the decoder, is a weight parameter used to weigh the importance of bit rate and reconstruction loss.

[0019] In step 4, during the training of the image encoding and decoding network, the latent features and super prior The tail variables of the image codec network are set to zero to enable the image codec network to obtain the ability of progressive coding. At the same time, the potential features of the non-tail are simulated by zeroing the data of some channels, that is, random masking of the channels, and the channel positions to be zeroed are selected based on the Gilbert-Elliott model. The present invention introduces the Gilbert-Elliott model, a two-state Markov method, to simulate the network environment of the real world. The model is widely used to characterize the error mode in lossy connected networks and analyze the performance of error detection and correction techniques. In a real transmission environment, when channel-level progressive coding is performed, packet loss is difficult to simulate, but usually presents a continuous cycle. This means that if one channel is lost, the probability of losing subsequent channels will also increase. The Gilbert-Elliott model can easily adapt to such simulations, while random masks cannot, which significantly reduces its loss recovery ability in actual network environments. On the other hand, considering that no grouping is performed during training, the channels are grouped and transmitted according to the bit rate constraint during transmission. Therefore, the probability of losing multiple channels in the same group is significantly increased. This scenario is consistent with the assumptions of the Gilbert-Elliott model and better reflects the potential packet loss that may occur.

[0020] In step 5, the compressed super prior is sent as a packet, and the potential features are grouped and packaged: first, each channel in the potential feature is compressed separately to obtain the estimated amount of information, and then the data in the potential feature channel is combined in the order of the channel index. When the current data packet size exceeds the maximum limit of the data packet due to the addition of the current channel data, the channel data before the current channel is regarded as a data packet. In the actual transmission process, there may be a slight deviation from the estimated information content. If the actual compressed size of the channel data in the data packet still exceeds the maximum limit of the data packet, the current data packet is divided into two data packets for transmission; finally, check whether the index of the last channel in the current data packet is divisible by 4. If so, move the last channel data of the current data packet to the next data packet. If not, no additional operation is performed. This method reduces the impact of packet loss without affecting the performance of progressive coding.

[0021] In the present invention, in order to distinguish between the double tail drop method in progressive coding and the channel random cover strategy for simulating network packet loss, different methods are used in the training and reasoning stages. During the training process, the tail drop method randomly selects a certain number of tail channels to be zeroed in each iteration, while the channel random cover strategy sets any channel to zero according to the selection of the distribution model. In the actual reasoning stage, the double tail drop method directly deletes the channels that do not need to be transmitted at the current level, while the channel random cover strategy sets the channels corresponding to the lost data packets to zero.

[0022] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0023] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is run on a computer, the steps of the method described are executed.

[0024] Beneficial effects: The present invention can achieve bit rate scalability in the channel dimension in extremely low bandwidth satellite networks based on the training perceptual progressive coding method. Given that each channel of the encoded features contains unique information that is critical to decoding, the present invention introduces a spatial channel rearrangement mechanism on the encoder side, which redistributes the information of each channel to multiple channels to prevent irreversible errors. On the decoder side, we adopt a mask conditional aggregation module to extract loss masks from the received incomplete features and use them together with the received features as additional conditional information for the decoder. This method enhances the reconstruction results through effective loss modeling. In addition, the packet loss simulation of the Gilbert-Elliott model is incorporated into the training process to improve the generalization ability of the model in actual network environments. Evaluations using common data sets show that the method of the present invention outperforms traditional methods and deep learning-based methods in both compression performance and packet loss stability. The algorithm of the present invention has good versatility in both simulation and actual network evaluation, and can prevent pixel errors or color deviations during progressive transmission. In addition, our method introduces acceptable computational complexity, making it suitable for real-world applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0026] Figure 1 It is a system diagram of the overall algorithm of the present invention.

[0027] Figure 2 This is a schematic diagram of the principle of spatial channel rearrangement.

[0028] Figure 3 It is a schematic diagram of the principle of the mask conditional aggregation module.

[0029] Figure 4 It is a performance comparison of the present invention on the Kodak data set on a real satellite channel. DETAILED DESCRIPTION

[0030] like Figure 1 As shown, an embodiment of the present invention provides a loss-resistant image coding method for an unstable satellite network, comprising the following steps:

[0031] Step 1, establishing an end-to-end image coding and decoding network, the image coding and decoding network includes a coding end and a decoding end, the coding end includes an encoder and a super-prior encoder, the decoding end includes a decoder and a super-prior decoder; a given input image is passed through the encoder to obtain compact potential features, the potential features are input into the super-prior encoder to obtain super-prior, the potential features and super-prior after entropy coding and quantization are binary code streams, after entropy decoding the binary code stream, the quantized super-prior is first input into the super-prior decoder, and the obtained output and the potential features are input into the decoder to complete decoding;

[0032] Step 2, such as Figure 2 As shown, during encoding, the latent features Rearrange the spatial channels and transform the potential features Divide into small cubes with a length of 2, a width of 2 and 4 channels. In each cube, put the data in the upper left corner of the cube into the first channel of the original cube in the order from left to right and from top to bottom. Similarly, the data in the upper right corner, lower left corner and lower right corner are rearranged to the second, third and fourth channels of the original cube respectively. After all the small cubes are rearranged, they are quantized to obtain , the corresponding inverse transformation is required during decoding. Considering the complexity, the present invention selects 4 consecutive channels as the window for spatial channel rearrangement. In the actual packaging process, the last channel in each data packet will be checked. If the number of channels is a multiple of 4, it will be moved to the next data packet. This method reduces the impact of packet loss without affecting the performance of progressive coding;

[0033] Step 3, such as Figure 3 As shown in Figure 1, a binary mask is used to indicate whether a channel of the latent feature is lost during transmission. A two-layer convolutional network is used to model the mask, and then the mask is concatenated with the latent feature to obtain the combined feature. Another convolutional layer and GELU activation function are used for feature fusion to input into the decoder;

[0034] Step 4: Use the double tail drop strategy, channel random masking strategy, and Gilbert-Elliott model to simulate the packet loss network to train the model. During the training process, the tail drop method randomly selects a certain number of tail channels to zero in each iteration, while the channel random masking strategy sets any channel to zero according to the selection of the distribution model. In the actual reasoning stage, the double tail drop method directly deletes the channels that do not need to be transmitted at the current level, while the channel random masking strategy sets the channels corresponding to the lost packets to zero.

[0035] Step 5, send the compressed hyper-prior as a packet and group the potential features. Grouping and packing first estimates the amount of information in each channel of the potential feature. Then, combine the data in the potential feature channels in the order of channel index. When adding the current channel data causes the current data packet size to exceed the maximum limit of the data packet, we regard the channel data before the current channel as one data packet. In the actual transmission process, there may be a slight deviation from the estimated information content. If the actual compressed size of the channel data in this data packet still exceeds the maximum limit of the data packet, the current data packet will be divided into two data packets for transmission. Finally, check whether the index of the last channel in the current data packet can be a multiple of 4. If so, move the last channel data of the current data packet to the next data packet, if not, do no additional operation.

[0036] The method provided in this embodiment has good versatility in both simulation and actual network evaluation, and can prevent pixel errors or color deviations during progressive transmission. Figure 4 As shown in the figure, the present method (Ours in the figure) has better anti-packet loss performance than the currently common progressive coding schemes. At the same or higher packet loss rate and the same bpp (bits per pixel), the present method has a higher PSNR (peak signal-to-noise ratio) than JPEG2000 and ProgDTD. The #p in the upper right corner of the picture indicates the reconstruction result after the decoder receives the pth data packet. The first column shows the initial results of the progressive transmission of the three methods without any packet loss. The second column shows the results when the third packet is lost. The third column shows the results when the sixth packet is also lost (for JPEG2000, only the third packet is lost because the subsequent packets of JPEG2000 cannot be used for decoding after the packet loss). The fourth column shows the decoding results of the packet loss after the transmission is completed. The method of the present invention always shows excellent performance. In addition, the method of the present invention introduces an acceptable computational complexity, making it suitable for real-world applications.

[0037] The present invention provides a method for anti-loss image coding for unstable satellite networks. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A loss-resistant image coding method for unstable satellite networks, characterized in that: The following steps are involved: Step 1, establishing an end-to-end image coding and decoding network, the image coding and decoding network includes a coding end and a decoding end, the coding end includes an encoder and a super-prior encoder, the decoding end includes a decoder and a super-prior decoder; a given input image is passed through the encoder to obtain compact potential features, the potential features are input into the super-prior encoder to obtain super-prior, the potential features and super-prior after entropy coding and quantization are binary code streams, after entropy decoding the binary code stream, the quantized super-prior is first input into the super-prior decoder, and the obtained output and the potential features are input into the decoder to complete decoding; Step 2: During encoding, the latent features extracted by the encoder from the image are spatially rearranged, and the rearranged latent features are then input into the super prior encoder for distribution modeling; Step 3: The decoder uses the mask to mark the potential features, uses a convolutional neural network to aggregate the potential features and the mask to obtain the combined features, and inputs the combined features into the decoder for decoding to obtain the reconstructed image; Step 4: During the image encoding and decoding network training process, the double tail drop strategy, channel random masking strategy, and Gilbert-Elliott model are used to simulate the packet loss network to train the model; Step 5: Pack and send the compressed hyper-prior and latent features in groups. When grouping, it is additionally determined whether the current channel sequence number is a multiple of 4. If it is, the current channel is divided into the next package to be sent. If not, no additional operation is performed.

2. The method according to claim 1, characterized in that: Step 2 includes: during encoding, spatial channel rearrangement of the latent feature y is performed, the latent feature y is divided into small cubes with a length of 2, a width of 2 and 4 channels, and in each cube, the data of the upper left corner of the cube is placed in the first channel of the original cube in the order from left to right and from top to bottom, and so on, the data of the upper right corner, the lower left corner, and the lower right corner are rearranged to the second, third, and fourth channels of the original cube respectively, and after all the small cubes are rearranged, the rearranged latent feature is obtained through quantization.

3. The method according to claim 2, characterized in that In step 3, use the binary mask m r ∈{0, 1} C To annotate the rearranged latent features Where C is the number of channels of potential features, m r is the mask of the rearranged latent features; if the mask of the ith rearranged latent feature Indicates that the i-th rearranged latent feature has been received if Indicates packet loss, the i-th rearranged latent feature is not received 1≤i≤C, the received incomplete features are derived by the following formula: where ⊙ represents element-wise multiplication, represents the residual rearranged latent features received by the decoder after packet loss; After the spatial channels are rearranged, the rearranged potential features are obtained The original arrangement of and the corresponding loss mask m.

4. The method according to claim 3, characterized in that: In step 3, two convolutional layers are deployed, each followed by a GELU activation function to model the loss mask m, aggregate the incomplete features conditioned on packet loss, and combine the loss mask m with the rearranged latent features. Connect to obtain the combined features, then use 1 convolution layer and 1 GELU activation function for feature fusion, and input the fused features into the decoder; at this time, the loss function L is: in Represents the quantized rearranged latent features The code rate, Represents the quantized hyper-prior The code rate, M represents the mask condition aggregation operation, g s Represents a decoder, represents the reconstruction loss between the input image x and the output of the decoder, and λ is a weight parameter used to weigh the importance of bitrate and reconstruction loss.

5. The method according to claim 4, characterized in that In step 4, during the training of the image encoding and decoding network, the tail variables of the latent feature y and the super prior z are set to zero to enable the image encoding and decoding network to obtain the ability of progressive encoding. At the same time, the non-tail latent features are again simulated by zeroing the data of some channels, that is, channel random masking, and the zeroed channel positions are selected based on the Gilbert-Elliott model.

6. The method according to claim 5, characterized in that In step 5, the compressed super-prior is sent as a packet, and the potential features are grouped and packaged: first, each channel in the potential feature is compressed separately to obtain the estimated amount of information, and then the data in the potential feature channel is combined in the order of channel index. When the addition of the current channel data causes the current data packet size to exceed the maximum limit of the data packet, the channel data before the current channel is regarded as a data packet. During the actual transmission process, if the actual compressed size of the channel data in the data packet still exceeds the maximum limit of the data packet, the current data packet is divided into two data packets for transmission; finally, check whether the index of the last channel in the current data packet is divisible by 4. If so, move the last channel data of the current data packet to the next data packet. If not, no additional operation is performed.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 6 are executed.

Images