Satellite communication-oriented deep learning progressive out-of-order transmission image compression method and device

Abstract

The application provides a satellite communication-oriented deep learning progressive out-of-order transmission image compression method and device, and relates to the field of image compression. The method comprises the following steps: a sending end extracts latent features of an original image to obtain a sending end latent feature quantization representation and corresponding sending end hyper-prior feature encoding quantization representation and sending end hyper-prior feature decoding quantization representation; the sending end hyper-prior feature encoding quantization representation is encoded to obtain a hyper-prior feature bit stream and sent to a receiving end; the sending end hyper-prior feature decoding quantization representation is processed to obtain sending end context parameters, and a variance of the sending end latent feature quantization representation is generated according to the sending end context parameters; at least according to the sending end context parameters, a mean of the sending end latent feature quantization representation is obtained; and according to the variance and the mean, the sending end latent feature quantization representation is encoded to obtain a latent feature bit stream and sent to the receiving end, so as to realize out-of-order transmission of image data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image compression, and in particular to a deep learning progressive out-of-order transmission image compression method and device for satellite communication. BACKGROUND

[0002] Image compression technology has important application value in resource-limited scenarios such as remote sensing monitoring and emergency communication. Narrowband communication systems represented by Beidou short message satellite communication have advantages such as wide coverage and strong anti-interference capability, and are suitable for image transmission tasks that cannot rely on ground infrastructure. However, due to the characteristics of low transmission rate, long data packet interval, and easy packet loss, image data often encounters serious bandwidth bottlenecks and reliability problems during transmission, which puts high requirements on compression efficiency and decoding robustness.

[0003] In order to adapt to bandwidth-limited environments, image compression methods have evolved from rule-driven to data-driven. In recent years, deep learning-based image compression methods have become a research hotspot, and through end-to-end optimization of reconstruction quality and compression rate, they have significantly surpassed traditional encoding methods. Under high delay and unstable channel conditions, progressive coding (Progressive Coding) or scalable coding (Scalable Coding) technology is proposed to enhance the adaptability of transmission. This kind of method encodes the image into a bit stream that can be decoded in layers, so that the receiving end can generate a low-quality preview image even if only part of the data packet is obtained, and gradually improve the reconstruction quality as the data increases. However, most methods assume that the communication process is error-free, and lack of structural design for high packet loss environments. In addition, the image transmission process in the satellite communication scenario also faces the actual constraints of long packet interval and limited frequency, which means that in non-progressive transmission systems, the receiving end needs to wait for a very long time to receive the complete image. This high delay greatly affects the image availability and decision-making efficiency in emergency communication and real-time monitoring tasks.

[0004] Therefore, for resource-limited and time-sensitive satellite communication scenarios, how to improve the robustness of progressive image compression and transmission is a technical problem that needs to be solved by the present application. SUMMARY

[0005] Based on the above technical problems, the present application provides a deep learning progressive out-of-order transmission image compression method and device for satellite communication, which aims to overcome the above problems or at least partially solve the above problems.

[0006] The first aspect of the present application provides a deep learning progressive out-of-order transmission image compression method for satellite communication, the method comprising:

[0007] The sending end in the semantic communication system extracts latent features of an original image through an encoder to obtain a sending end latent feature quantized representation and corresponding sending end hyper-prior feature encoded quantized representation and sending end hyper-prior feature decoded quantized representation;

[0008] The sending end encodes the sending end hyper-prior feature encoded quantized representation through a first arithmetic encoder to obtain a hyper-prior feature bit stream, and sends the hyper-prior feature bit stream to a receiving end in the semantic communication system;

[0009] The sending end processes the sending end hyper-prior feature decoded quantized representation through a first synthesis transformation module to obtain sending end context parameters, and generates a variance of the sending end latent feature quantized representation according to the sending end context parameters;

[0010] The sending end obtains a mean of the sending end latent feature quantized representation through a first entropy model according to at least the sending end context parameters;

[0011] The sending end encodes the sending end latent feature quantized representation through a second arithmetic encoder according to the variance and the mean of the sending end latent feature quantized representation to obtain a latent feature bit stream, and sends the latent feature bit stream to the receiving end.

[0012] The second aspect of the present application provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is executed by the processor to realize the satellite communication oriented deep learning progressive out-of-order transmission image compression method according to the first aspect of the present application.

[0013] In the satellite communication-oriented deep learning progressive out-of-order transmission image compression method, the feature distribution parameters (mean and variance) output by the traditional entropy model are decoupled to propose a new paradigm of progressive image compression transmission: for the variance, the sending end hyper-prior feature decoding quantization representation is processed to obtain the sending end context parameter, and the variance of the sending end latent feature quantization representation is obtained according to the sending end context parameter; for the mean, the first entropy model at least processes the sending end context parameter to obtain the mean of the sending end latent feature quantization representation. In this way, the generation of each compression package in the image transmission process no longer depends on other packages, breaking the order dependence between packages, realizing arbitrary order packaging and out-of-order transmission of image data, enhancing the adaptability of image compression transmission to high delay links, so that for the narrowband and long delay satellite communication environment, the technical bottlenecks of the traditional progressive image compression method in the aspects of package coupling, high compression rate information concentration and sequential transmission dependence can be effectively alleviated, and the transmission efficiency and robustness of the image compression system (i.e. the above semantic communication system) under complex links such as high packet loss, high delay and low bandwidth are improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 is a step flow chart of a satellite communication-oriented deep learning progressive out-of-order transmission image compression method according to an embodiment of the present application;

[0016] Figure 2 is a channel segmentation schematic diagram according to an embodiment of the present application;

[0017] Figure 3 is a spatial segmentation method schematic diagram according to an embodiment of the present application;

[0018] Figure 4 is a spatial channel autoregressive processing schematic diagram according to an embodiment of the present application;

[0019] Figure 5 is a sending end processing process schematic diagram according to an embodiment of the present application;

[0020] Figure 6 is a receiving end processing process schematic diagram according to an embodiment of the present application;

[0021] Figure 7This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of a deep learning-based progressive out-of-order transmission image compression method for satellite communication, as shown in an embodiment of the present invention. Figure 1 As shown, the deep learning-based progressive out-of-order transmission image compression method for satellite communication provided in this embodiment includes at least the following steps:

[0024] Step S11: In the semantic communication system, the sending end extracts the latent features of the original image through the encoder to obtain the quantized representation of the latent features of the sending end and the corresponding quantized representation of the encoded and decoded features of the sending end.

[0025] In this embodiment, the semantic communication system includes a transmitter and a receiver. The transmitter includes at least an encoder, a first arithmetic encoder, a first synthesis transform module, a first entropy model, and a second arithmetic encoder. The encoder is used to extract latent features from the original image. The original image can be input into the encoder in the semantic communication system to obtain the latent features of the original image output by the encoder. The original image includes: an image to be transmitted in a satellite communication scenario, or other images to be transmitted with high requirements for transmission bandwidth and robustness; there are no restrictions on this. The latent features characterize semantic features. The semantic communication system in this embodiment can be an image compression system, designed to achieve compressed transmission of images through semantic communication.

[0026] In this embodiment, after obtaining the latent features of the original image, the latent feature quantization representation of the transmitter can be obtained based on the latent features of the original image; the corresponding super-prior feature encoding quantization representation of the transmitter can be obtained based on the latent features of the original image; and the corresponding super-prior feature decoding quantization representation of the transmitter can be obtained based on the latent features of the original image.

[0027] Step S12: The sending end encodes the quantized representation of the sending end's prior feature encoding through the first arithmetic encoder to obtain the prior feature bit stream, and sends the prior feature bit stream to the receiving end in the semantic communication system.

[0028] In this embodiment, after obtaining the sender hyper-prior feature encoding quantization representation, the sender can encode the sender hyper-prior feature encoding quantization representation through the first arithmetic encoder to obtain a hyper-prior feature bit stream output by the first arithmetic encoder, and send the hyper-prior feature bit stream to the receiver in the semantic communication system.

[0029] Step S13: The sender processes the sender hyper-prior feature decoding quantization representation through the first synthesis transformation module to obtain a sender context parameter, and generates a variance of the sender latent feature quantization representation according to the sender context parameter.

[0030] In this embodiment, after obtaining the sender hyper-prior feature decoding quantization representation, the sender can process the sender hyper-prior feature decoding quantization representation through the first synthesis transformation module to obtain a sender context parameter, and then generate a variance of the sender latent feature quantization representation through the first synthesis transformation module according to the obtained sender context parameter.

[0031] Step S14: The sender obtains a mean of the sender latent feature quantization representation through the first entropy model according to at least the sender context parameter.

[0032] In this embodiment, after obtaining the sender context parameter, the sender can input at least the sender context parameter into the first entropy model to obtain a mean of the sender latent feature quantization representation through the first entropy model.

[0033] Step S15: The sender encodes the sender latent feature quantization representation through the second arithmetic encoder according to the variance and the mean of the sender latent feature quantization representation, obtains a latent feature bit stream, and sends the latent feature bit stream to the receiver.

[0034] In this embodiment, after obtaining the variance and the mean of the sender latent feature quantization representation through the first synthesis transformation module and the first entropy model respectively, the sender can encode the sender latent feature quantization representation through the second arithmetic encoder deployed in the sender based on the variance and the mean of the sender latent feature quantization representation, obtain a latent feature bit stream output by the second arithmetic encoder, and send the latent feature bit stream to the receiver in the semantic communication system, so that the receiver can decode and reconstruct at least based on the latent feature bit stream and the hyper-prior feature bit stream to obtain a reconstructed image corresponding to the original image.

[0035] In the embodiment, the sending end decouples the feature distribution parameters (mean and variance) output by the traditional entropy model to propose a new paradigm for progressive image compression transmission: for the variance, the sending end context parameter is obtained by processing the sending end hyper-prior feature quantization representation, and the variance of the sending end latent feature quantization representation is obtained according to the sending end context parameter; for the mean, the sending end latent feature quantization representation mean is obtained by processing at least the sending end context parameter through the first entropy model. In this way, the embodiment can make the generation of each compressed package no longer dependent on other packages based on the decoupled mean and variance during image transmission, break the order dependence between packages, realize arbitrary order packaging and out-of-order transmission of image data, enhance the adaptability of image compression transmission to high delay links, and thus for the narrowband and long delay satellite communication environment, the technical bottlenecks of the traditional progressive image compression method in aspects such as package coupling, high compression rate information concentration and sequential transmission dependence can be effectively alleviated, and the transmission efficiency and robustness of the image compression system (i.e. the above semantic communication system) under complex links such as high packet loss, high delay and low bandwidth are improved.

[0036] In combination with the above embodiments, in an implementation manner, the present application further provides a deep learning progressive out-of-order transmission image compression method for satellite communication, in which, in addition to the above steps, steps S21 to S23 can be further included:

[0037] Step S21: The receiving end processes the receiving end hyper-prior feature decoding quantization representation through the second synthesis transformation module to obtain the receiving end context parameter, and generates the variance of the receiving end latent feature quantization representation corresponding to the receiving end received latent feature bit stream according to the receiving end context parameter.

[0038] In the embodiment, the receiving end at least includes a first arithmetic decoder, a third arithmetic decoder, a second entropy model and a second synthesis transformation module. The receiving end can receive the hyper-prior feature bit stream and the latent feature bit stream sent by the sending end to obtain the received hyper-prior feature bit stream and the received latent feature bit stream, respectively.

[0039] The receiving end can decode the received hyper-prior feature bit stream through the third arithmetic decoder to obtain a receiving end hyper-prior feature decoded quantized representation. That is, the receiving end hyper-prior feature decoded quantized representation is obtained by decoding the received hyper-prior feature bit stream by the receiving end. Then, the receiving end can process the receiving end hyper-prior feature decoded quantized representation through the second synthesis transformation module to obtain a receiving end context parameter, and generate a variance of a receiving end latent feature quantized representation according to the obtained receiving end context parameter through the second synthesis transformation module. The receiving end latent feature quantized representation is a latent feature quantized representation corresponding to the received latent feature bit stream by the receiving end.

[0040] It can be understood that the first synthesis transformation module and the second synthesis transformation module are the same, and the method of generating the sending end context parameter and the variance of the sending end latent feature quantized representation by the first synthesis transformation module is the same as the method of generating the receiving end context parameter and the variance of the receiving end latent feature quantized representation by the second synthesis transformation module.

[0041] Step S22: The receiving end obtains a mean of the receiving end latent feature quantized representation through a second entropy model according to at least the receiving end context parameter.

[0042] In this embodiment, after obtaining the receiving end context parameter, the receiving end can input at least the receiving end context parameter to the second entropy model to obtain the mean of the receiving end latent feature quantized representation through the second entropy model. The first entropy model and the second entropy model are the same.

[0043] Step S23: The receiving end decodes the received latent feature bit stream through the first arithmetic decoder according to the variance and the mean of the receiving end latent feature quantized representation to obtain a reconstructed image.

[0044] In this embodiment, after obtaining the variance and the mean of the receiving end latent feature quantized representation, the receiving end can decode the received latent feature bit stream based on the variance and the mean of the receiving end latent feature quantized representation through the first arithmetic decoder to obtain a decoded receiving end latent feature quantized representation, and further perform semantic decoding on the decoded receiving end latent feature quantized representation to obtain a reconstructed image corresponding to the original image.

[0045] In the embodiment, similar to the sending end, the receiving end also decouples the feature distribution parameters (mean and variance) output by the traditional entropy model, and proposes a new paradigm for progressive image compression transmission: for the variance, the receiving end context parameter is obtained by processing the decoding and quantization representation of the receiving end hyper-prior feature, and the variance of the receiving end latent feature quantization representation is obtained according to the receiving end context parameter; for the mean, the mean of the receiving end latent feature quantization representation is obtained by processing at least the receiving end context parameter through the second entropy model. In this way, the embodiment can make the generation of each compressed package no longer dependent on other packages in the image transmission process based on the decoupled mean and variance, break the order dependence between packages, realize arbitrary order packaging and out-of-order transmission of image data, enhance the adaptability of image compression transmission to high delay links, and thus for the narrowband and long delay satellite communication environment, the technical bottlenecks of the traditional progressive image compression method in the aspects of package coupling, high compression rate information concentration and sequential transmission dependence can be effectively alleviated, and the transmission efficiency and robustness of the image compression system (i.e. the above semantic communication system) under complex links such as high packet loss, high delay and low bandwidth are improved.

[0046] In combination with any of the above embodiments, the application further provides a deep learning progressive out-of-order transmission image compression method for satellite communication. In the method, in addition to the above steps, step S31 can be further included, and step S22 can specifically include step S32.

[0047] Step S31: In the case where the receiving end does not receive the encoding result of the nth sending end latent feature quantization representation channel slice in the latent feature bit stream, the mean of the nth receiving end latent feature quantization representation channel slice is taken as the encoding result of the received nth receiving end latent feature quantization representation channel slice.

[0048] In the embodiment, the latent feature bit stream includes the encoding results of the 1st to Nth sending end latent feature quantization representation channel slices, and N is an integer greater than 1. It should be noted that the encoding result of the 1st sending end latent feature quantization representation channel slice in the latent feature bit stream is received by the receiving end as the encoding result of the 1st receiving end latent feature quantization representation channel slice received by the receiving end; …, the encoding result of the Nth sending end latent feature quantization representation channel slice in the latent feature bit stream is received by the receiving end as the encoding result of the Nth receiving end latent feature quantization representation channel slice received by the receiving end.

[0049] In the image transmission process, the receiving end can not receive or temporarily cannot receive the encoding result of the potential feature quantization representation channel slice of a certain sending end due to various reasons. In the case that the receiving end does not receive the encoding result of the nth receiving end potential feature quantization representation channel slice in the potential feature bit stream, the receiving end can take the mean value of the nth receiving end potential feature quantization representation channel slice obtained by the receiving end through the second entropy model as the encoding result of the received nth receiving end potential feature quantization representation channel slice. The nth receiving end potential feature quantization representation channel slice is the nth channel slice corresponding to the receiving end potential feature quantization representation.

[0050] Step S32: The receiving end obtains the mean value of the nth receiving end potential feature quantization representation channel slice through the second entropy model according to the receiving end context parameter and the encoding result of each of the received 1st to (n-1)th receiving end potential feature quantization representation channel slices.

[0051] In this embodiment, the receiving end can input the receiving end context parameter and the encoding result of each of the received 1st to (n-1)th receiving end potential feature quantization representation channel slices into the second entropy model, obtain the mean value of the nth receiving end potential feature quantization representation channel slice through the second entropy model, and n is an integer from 2 to N.

[0052] For the mean value of the 1st receiving end potential feature quantization representation channel slice, the receiving end inputs the receiving end context parameter into the second entropy model, and obtains the mean value of the 1st receiving end potential feature quantization representation channel slice through the second entropy model.

[0053] In the embodiment, since the variance of each latent feature quantization representation channel slice needs to be predicted by the encoding state of the previous one or more channel slices, if the encoding result of the previous channel slice is not successfully received or decoding fails, it will cause variance decoding error. Since the variance is a key parameter for constructing the probability model in the entropy model, and the entropy decoding process is lossless compression, even a slight deviation in the variance will cause serious decoding errors, which will lead to the decoding failure of all subsequent multiple compression packets, resulting in serious error propagation problem. However, the mean is not so critical, and inaccurate estimation of the mean will not cause serious decoding errors. Therefore, the embodiment decouples the variance and the mean, realizes the context independence of the latent feature decoding process, and on this basis, the probability modeling of each latent feature channel slice in the subsequent only depends on the fixed variance and part of the context state, without relying on the correct decoding result of the previous channel slice. This structural design greatly weakens the dependence between each compression packet, thereby realizing the out-of-order receiving and decoding of the compression bit stream, supporting more flexible packet assembly scheduling strategy, and improving the transmission efficiency and robustness of the system under the actual satellite communication link. That is, when part of the latent feature bit stream is lost during image transmission, and the receiving end cannot obtain the encoding result of the nth sending end latent feature quantization representation channel slice, the receiving end can directly replace the missing nth sending end latent feature quantization representation channel slice with the estimated mean of the nth receiving end latent feature quantization representation channel slice as the received encoding result of the nth receiving end latent feature quantization representation channel slice, thereby completing the recovery of the latent feature. The embodiment essentially utilizes the prediction ability of the super-prior probability prediction for the un-received information, realizes the automatic image semantic feature completion when the data packet is lost, without retransmission, and effectively alleviates the technical bottleneck of the related technology in the aspects of packet assembly coupling, high compression rate information set, and sequential transmission dependence.

[0054] In combination with any of the above embodiments, in an implementation, the present application further provides a deep learning progressive out-of-order transmission image compression method for satellite communication. In the method, after the step S31, the following steps S41 to S42 can be further included:

[0055] Step S41: In the case that the receiving end receives the encoding result of the nth sending end latent feature quantization representation channel slice in the latent feature bit stream, the receiving end re-obtains the mean of the (n+1)th receiving end latent feature quantization representation channel slice according to the receiving end context parameters and the respective encoding results of the first to nth receiving end latent feature quantization representation channel slices through the second entropy model.

[0056] In the actual communication application, especially in the satellite short message communication, there are characteristics such as large link delay, limited bandwidth, and long packet interval. Therefore, in the image compression and transmission process, supporting flexible packet assembly and out-of-order transmission becomes a key technical requirement. On the one hand, flexible packet assembly can dynamically determine the division and priority of data according to task requirements and channel state; on the other hand, out-of-order transmission helps to improve channel utilization and anti-packet loss robustness.

[0057] Based on this, after the receiving end receives the encoding result of the nth receiver-side latent feature quantization channel slice in the latent feature bit stream, and performs correlation processing on the basis of the mean value of the nth receiver-side latent feature quantization channel slice and the encoding result of the nth receiver-side latent feature quantization channel slice, the receiving end can obtain the mean value of the (n+1)th receiver-side latent feature quantization channel slice again through the second entropy model according to the receiving end context parameter and the encoding results of the first to nth receiver-side latent feature quantization channel slices.

[0058] Step S42: The receiving end redecodes the encoding result of the (n+1)th receiver-side latent feature quantization channel slice through the first arithmetic decoder according to the variance and the obtained mean value of the (n+1)th receiver-side latent feature quantization channel slice, to obtain an updated reconstructed image.

[0059] In the embodiment, the receiving end can redecode the encoding result of the (n+1)th receiver-side latent feature quantization channel slice through the first arithmetic decoder according to the variance of the (n+1)th receiver-side latent feature quantization channel slice and the obtained mean value of the (n+1)th receiver-side latent feature quantization channel slice, to obtain an updated reconstructed image.

[0060] In the embodiment, after the receiving end receives the respective encoding results of the nth receiver-side latent feature quantization channel slices, the receiving end can recompute the mean value on the basis of the received respective encoding results of the nth receiver-side latent feature quantization channel slices, and then redecode on the basis of the variance and the recomputed mean value, to further obtain a reconstructed image of the original image.

[0061] In combination with any of the above embodiments, in an implementation, the present application further provides a deep learning progressive out-of-order transmission image compression method for satellite communication. In the method, the above S11 can specifically include steps S51 to S54:

[0062] Step S51: The sender maps the latent features of the original image into the sender latent feature quantized representation through the first quantization module.

[0063] In this embodiment, the sender further comprises a first quantization module, a second quantization module, an analysis transformation module and a second arithmetic decoder. The sender can map the latent features of the original image into the sender latent feature quantized representation through the first quantization module.

[0064] Step S52: The sender extracts the hyper-prior features of the original image from the latent features of the original image through the analysis transformation module, and maps the hyper-prior features of the original image into the sender hyper-prior feature encoded quantized representation through the second quantization module.

[0065] In this embodiment, the sender can process the latent features of the original image through the analysis transformation module to extract the hyper-prior features of the original image from the latent features of the original image. The analysis transformation module (i.e. the hyper-prior module) is used to extract the main information of the original image using fewer bits, that is, the hyper-prior features of the original image represent the main information of the original image, that is, the minimum information necessary for the reconstructed image. After obtaining the hyper-prior features, the sender can map the hyper-prior features of the original image into the sender hyper-prior feature encoded quantized representation through the second quantization module.

[0066] Step S53: The sender saves the hyper-prior feature bit stream locally.

[0067] In this embodiment, after obtaining the hyper-prior feature bit stream based on the sender hyper-prior feature encoded quantized representation, the sender saves the hyper-prior feature bit stream locally in addition to sending the hyper-prior feature bit stream to the receiver.

[0068] Step S54: The sender decodes the hyper-prior feature bit stream through the second arithmetic decoder to obtain the sender hyper-prior feature decoded quantized representation.

[0069] In this embodiment, the sender can decode the hyper-prior feature bit stream through the second arithmetic decoder to obtain the sender hyper-prior feature decoded quantized representation.

[0070] In combination with any of the above embodiments, in an implementation, the application further provides a satellite communication oriented deep learning progressive out-of-order transmission image compression method. In the method, after step S51, step S61 can be further included, and the "generating the variance of the sender latent feature quantized representation according to the sender context parameter" in step S13 can specifically include step S62, and the "encoding the sender latent feature quantized representation by the second arithmetic encoder according to the variance and the mean of the sender latent feature quantized representation to obtain a latent feature bit stream" in step S15 can specifically include step S63:

[0071] Step S61: The sender performs channel segmentation on the sender latent feature quantized representation to obtain N sender latent feature quantized representation channel slices.

[0072] In the embodiment, the sender can perform channel segmentation on the sender latent feature quantized representation to obtain N sender latent feature quantized representation channel slices, where N is an integer greater than 1. In an example, as shown in Figure 2 Figure 2 is a channel segmentation schematic diagram shown by an embodiment of the application. In Figure 2 , the sender latent feature quantized representation is divided into three parts by channel segmentation, for example, from front to back, the first row is the first sender latent feature quantized representation channel slice, the second row is the second sender latent feature quantized representation channel slice, and the third row is the third sender latent feature quantized representation channel slice.

[0073] Step S62: The sender generates the variance of the nth sender latent feature quantized representation channel slice according to the sender context parameter.

[0074] In the embodiment, the sender can generate the variance of the nth sender latent feature quantized representation channel slice according to the sender context parameter by the first synthesis transformation module, where n is an integer from 1 to N.

[0075] Step S63: The sender encodes the nth sender latent feature quantized representation channel slice by the second arithmetic encoder according to the variance and the mean of the nth sender latent feature quantized representation channel slice to obtain the encoding result of the nth sender latent feature quantized representation channel slice.

[0076] ​In this embodiment, the sending end can obtain the variance of the nth sending end latent feature quantization representation channel slice and the mean value of the nth sending end latent feature quantization representation channel slice according to the obtained nth sending end latent feature quantization representation channel slice, encode the nth sending end latent feature quantization representation channel slice through the second arithmetic encoder, and obtain the encoding result of the nth sending end latent feature quantization representation channel slice, so as to obtain the encoding results of the first to nth sending end latent feature quantization representation channel slices. In this embodiment, the encoding results of the first to nth sending end latent feature quantization representation channel slices constitute the latent feature bit stream.

[0077] In combination with the above embodiments, in an embodiment, similar to the processing process of the sending end, at the receiving end, the received latent feature bit stream includes the encoding results of the first to nth receiving end latent feature quantization representation channel slices. The receiving end can generate the variance of the receiving end latent feature quantization representation corresponding to the encoding result of the first to nth receiving end latent feature quantization representation channel slice according to the receiving end context parameter, and obtain the mean value of the receiving end latent feature quantization representation corresponding to the encoding result of the first to nth receiving end latent feature quantization representation channel slice through the second entropy model according to at least the receiving end context parameter, so as to decode the encoding result of the nth receiving end latent feature quantization representation channel slice through the first arithmetic encoder according to the variance and the mean value of the nth receiving end latent feature quantization representation channel slice, obtain the decoding result of the encoding result of the nth receiving end latent feature quantization representation channel slice (that is, obtain the decoded nth receiving end latent feature quantization representation channel slice), and further obtain the reconstructed image.

[0078] In combination with any of the above embodiments, in an embodiment, the present application also provides a deep learning progressive out-of-order transmission image compression method for satellite communication. In the method, the step S14 can specifically include steps S71 to S72:

[0079] Step S71: The sending end obtains the mean value of the first sending end latent feature quantization representation channel slice through the first entropy model according to the sending end context parameter.

[0080] In this embodiment, for the first sending end latent feature quantization representation channel slice, the sending end can send the sending end context parameter to the first entropy model to obtain the mean value of the first sending end latent feature quantization representation channel slice through the first entropy model.

[0081] Step S72: The sending end obtains the mean value of the nth sending end latent feature quantization representation channel slice through the first entropy model according to the sending end context parameter and the encoding results of the first to (n-1)th sending end latent feature quantization representation channel slices.

[0082] In the embodiment, for the nth sending end latent feature quantization representation channel slice, n is an integer from 2 to N, the sending end inputs the sending end context parameter and the encoding results of the first to the (n-1)th sending end latent feature quantization representation channel slices into the first entropy model, and obtains the mean value of the nth sending end latent feature quantization representation channel slice through the first entropy model.

[0083] In an embodiment, the entropy model (i.e., the first entropy model and the second entropy model) in the embodiment is an autoregressive model (Entropy Model), which is used to predict the to-be-encoded content (the mean value of the nth sending end latent feature quantization representation channel slice) according to the encoded content (the encoding results of the first to the (n-1)th sending end latent feature quantization representation channel slices), and the model supports progressive coding.

[0084] In combination with the above embodiments, in an embodiment, similar to the processing process of the sending end, at the receiving end, for the first receiving end latent feature quantization representation channel slice, the receiving end can send the receiving end context parameter to the second entropy model, and obtain the mean value of the first receiving end latent feature quantization representation channel slice through the second entropy model. For the nth receiving end latent feature quantization representation channel slice, n is an integer from 2 to N, the receiving end inputs the receiving end context parameter and the decoding results corresponding to the encoding results of the first to the (n-1)th receiving end latent feature quantization representation channel slices into the second entropy model, and obtains the mean value of the nth sending end latent feature quantization representation channel slice through the second entropy model.

[0085] In combination with any of the above embodiments, in an implementation, the present application further provides a deep learning progressive out-of-order transmission image compression method for satellite communication. In the method, after the above S61, the method can further include a step S81, and the above step S71 can specifically include a step S82, the above step S72 can specifically include a step S83 and a step S84.

[0086] Step S81: According to the checkerboard mode, the sending end performs spatial division on the nth sending end latent feature quantization representation channel slice to obtain the first to the Nth spatial slices corresponding to the nth sending end latent feature quantization representation channel slice. spatial slices and the (N+1)th spatial slices of the nth sending end latent feature quantization representation channel slice. n is an integer from 1 to N.

[0087] In the embodiment, after the channel division, the checkerboard division is further performed. Specifically, according to the checkerboard mode, each channel slice is divided into two parts: the sending end performs spatial division on the nth sending end latent feature quantization representation channel slice to obtain the first to the Nth spatial slices corresponding to the nth sending end latent feature quantization representation channel slice. spatial slices and the (N+1)th spatial slices of the nth sending end latent feature quantization representation channel slice. Spatial slice, n is an integer from 1 to N.

[0088] In an example, as Figure 3 shown, Figure 3 is a schematic diagram of a spatial slicing method according to an embodiment of the present application. In Figure 3 the left side of the figure is the i-th sender latent feature quantized representation channel slice, and the right side of the figure is the i-th sender latent feature quantized representation channel slice corresponding to the i-th spatial slice. Spatial slice, n is an integer from 1 to N. In this embodiment, the superscript represents the channel slice index, and the subscript represents the spatial slice index.

[0089] Step S82: The sender obtains the mean of the i-th spatial slice according to the sender context parameter and the encoding result of the i-th spatial slice through the spatial module in the first entropy model. Step S82: The sender obtains the mean of the i-th spatial slice according to the sender context parameter and the encoding result of the i-th spatial slice through the spatial module in the first entropy model.

[0090] In this embodiment, the first entropy model is divided into a channel module and a spatial module. For the first sender latent feature quantized representation channel slice, the sender can input the sender context parameter into the channel module in the first entropy model to obtain the mean of the i-th spatial slice through the channel module in the first entropy model.

[0091] Then, the sender inputs the sender context parameter and the encoding result of the i-th spatial slice into the spatial module in the first entropy model to obtain the mean of the i-th spatial slice through the spatial module in the first entropy model. In this way, the mean of the i-th spatial slice and the mean of the i-th spatial slice constitute the mean of the first sender latent feature quantized representation channel slice.

[0092] Step S83: The sender obtains the mean of the i-th spatial slice according to the sender context parameter and the encoding result of the first to the n-1-th sender latent feature quantized representation channel slice through the channel module in the first entropy model.

[0093] ​​​​​​​​​​​In this embodiment, for the n-th sending-end latent feature quantized representation channel slice, the sending end can input the sending-end context parameter and the encoding result of each of the 1st to the (n-1)th sending-end latent feature quantized representation channel slice into the channel module in the first entropy model, and obtain the mean value of the n-th sending-end latent feature quantized representation channel slice through the channel module in the first entropy model. the mean value of the spatial slice.

[0094] Step S84: The sending end obtains the mean value of the n-th sending-end latent feature quantized representation channel slice through the spatial module in the first entropy model according to the sending-end context parameter and the encoding result of the n-th sending-end latent feature quantized representation channel slice. the mean value of the spatial slice. the mean value of the spatial slice.

[0095] In this embodiment, the sending end inputs the sending-end context parameter and the encoding result of the 1st to the (n-1)th sending-end latent feature quantized representation channel slice into the spatial module in the first entropy model, and obtains the mean value of the n-th sending-end latent feature quantized representation channel slice through the spatial module in the first entropy model. the mean value of the spatial slice. the mean value of the spatial slice. In this way, the mean value of the n-th sending-end latent feature quantized representation channel slice is composed of the mean value of the n-th sending-end latent feature quantized representation channel slice and the mean value of the n-th sending-end latent feature quantized representation channel slice. the mean value of the spatial slice. the mean value of the spatial slice. In this way, the mean value of the n-th sending-end latent feature quantized representation channel slice is composed of the mean value of the n-th sending-end latent feature quantized representation channel slice and the mean value of the n-th sending-end latent feature quantized representation channel slice.

[0096] In combination with the above embodiments, in an embodiment, similar to the processing process of the sending end, at the receiving end, for the 1st receiving-end latent feature quantized representation channel slice, the receiving end can input the receiving-end context parameter into the channel module in the second entropy model, and obtain the mean value of the 1st spatial slice of the 1st receiving-end latent feature quantized representation channel slice through the channel module in the second entropy model.

[0097] Then, the receiving end inputs the receiving-end context parameter and the decoding result of the encoding result of the 1st spatial slice of the 1st receiving-end latent feature quantized representation channel slice (i.e., the 1st spatial slice of the 1st sending-end latent feature quantized representation channel slice sent by the sending end) received into the spatial module in the second entropy model, and obtains the mean value of the 2nd spatial slice of the 1st receiving-end latent feature quantized representation channel slice through the spatial module in the second entropy model. In this way, the mean value of the 1st spatial slice of the 1st receiving-end latent feature quantized representation channel slice and the mean value of the 2nd spatial slice of the 1st receiving-end latent feature quantized representation channel slice constitute the mean value of the 1st receiving-end latent feature quantized representation channel slice.

[0098] For the nth receiver latent feature quantization representation channel slice, where n is an integer from 2 to N, the receiver can input the receiver context parameters and the decoding results corresponding to the encoding results of the 1st to n-1th receiver latent feature quantization representation channel slices into the channel module of the second entropy model. Through the channel module of the second entropy model, the mean of the first spatial slice of the nth receiver latent feature quantization representation channel slice is obtained.

[0099] The receiver inputs the decoding result of the encoded result of the receiver context parameters and the first spatial slice of the nth receiver latent feature quantization representation channel slice (i.e., the first spatial slice of the nth transmitter latent feature quantization representation channel slice sent by the transmitter) into the spatial module of the second entropy model. Through the spatial module of the second entropy model, the mean of the second spatial slice of the nth receiver latent feature quantization representation channel slice is obtained. Thus, the mean of the first spatial slice of the nth receiver latent feature quantization representation channel slice and the mean of the second spatial slice of the nth receiver latent feature quantization representation channel slice constitute the mean of the nth receiver latent feature quantization representation channel slice. In this step, n is an integer from 2 to N.

[0100] In one embodiment, such as Figure 4 As shown, Figure 4 This is a schematic diagram illustrating a spatial channel autoregression process according to an embodiment of the present invention. Figure 4 In this example, the first entropy model is used. and These are the channel module and spatial module of the first entropy model, respectively. The second entropy model is the same as the first entropy model, and the processing is similar, so it will not be described in detail here. Specifically, the channel slice is represented by the quantization of the latent features of the i-th transmitter. You can first set the sender context parameters and The encoding results (i.e., the encoding results of the latent feature quantization representation channels from the 1st to the (i-1th)th transmitter) are passed through the channel module to obtain the first... Spatial slice mean Then, according to the first Spatial slice mean The quantization representation of the latent features of the i-th transmitter represents the channel slice. variance For the Spatial slice Encode to obtain the first Spatial slice The encoding result. Next, based on the sender context parameters... and the Spatial slice The encoding result, through the spatial module, yields the first... Spatial slice mean In this embodiment, the superscript represents the channel slice index, and the subscript represents the spatial slice index.

[0101] In conjunction with any of the above embodiments, in one implementation, the present invention also provides a deep learning-based progressive out-of-order transmission image compression method for satellite communication. In this method, step S63 may specifically include steps S91 and S92:

[0102] Step S91: The sending end, according to the first... The mean of the spatial slice and the variance of the channel slice are represented by the latent feature quantization of the nth transmitter. This variance is then processed by the second arithmetic encoder. Encoding spatial slices yields the first... Encoding results of spatial slices.

[0103] In this embodiment, the sending end receives the first... After averaging the spatial slices, we can proceed according to the... The mean of the spatial slice and the latent feature quantization of the nth transmitter represent the variance of the channel slice, which is then processed by the second arithmetic encoder. Encoding spatial slices yields the first... The encoding result of spatial slicing. Specifically, the variance of the channel slice is represented by the quantization of the latent features of the nth transmitter. It can be based on the sender context parameters The context parameters of the nth transmitter corresponding to the channel slice are represented by the latent feature quantization representation of the nth transmitter. Generate the variance of the channel slice by quantizing the latent features of the nth transmitter. Among them, the sending context parameters include: .

[0104] Step S92: The sending end, according to the first... The mean of the spatial slice and the variance of the channel slice are represented by the latent feature quantization of the nth transmitter. This variance is then processed by the second arithmetic encoder. Encoding spatial slices yields the first... The encoding result of the spatial slice, the first The encoding results of spatial slices and the first The encoding results of the spatial slice constitute the encoding results of the channel slice of the nth transmitter latent feature quantization representation.

[0105] In this embodiment, the sending end can, according to the first The mean of the spatial slice quantizes the variance of the nth transmitter latent feature quantized representation channel slice, and the first arithmetic encoder encodes the nth transmitter latent feature quantized representation channel slice based on the mean of the first spatial slice of the nth transmitter latent feature quantized representation channel slice and the variance of the nth transmitter latent feature quantized representation channel slice, to obtain the encoding result of the first spatial slice of the nth transmitter latent feature quantized representation channel slice. The encoding result of the spatial slice, so that the encoding result of the first spatial slice of the nth transmitter latent feature quantized representation channel slice and the encoding result of the second spatial slice of the nth transmitter latent feature quantized representation channel slice constitute the encoding result of the nth transmitter latent feature quantized representation channel slice. The encoding result of the spatial slice, so that the encoding result of the first spatial slice of the nth transmitter latent feature quantized representation channel slice and the encoding result of the second spatial slice of the nth transmitter latent feature quantized representation channel slice constitute the encoding result of the nth transmitter latent feature quantized representation channel slice.

[0106] In combination with the above embodiments, in an embodiment, similar to the processing process of the transmitter, at the receiver, after obtaining the mean of the first spatial slice of the nth receiver latent feature quantized representation channel slice, the receiver can decode the encoding result of the first spatial slice of the nth receiver latent feature quantized representation channel slice based on the mean of the first spatial slice of the nth receiver latent feature quantized representation channel slice and the variance of the nth receiver latent feature quantized representation channel slice through the first arithmetic decoder, to obtain the decoding result corresponding to the encoding result of the first spatial slice of the nth receiver latent feature quantized representation channel slice. Wherein, the variance of the nth receiver latent feature quantized representation channel slice can be generated according to the nth receiver context parameter in the receiver context parameter Wherein, the transmitter context parameter includes: , , .

[0107] The receiver can decode the encoding result of the second spatial slice of the nth receiver latent feature quantized representation channel slice based on the mean of the second spatial slice of the nth receiver latent feature quantized representation channel slice and the variance of the nth receiver latent feature quantized representation channel slice through the first arithmetic decoder, to obtain the decoding result corresponding to the encoding result of the second spatial slice of the nth receiver latent feature quantized representation channel slice, so that the decoding result corresponding to the encoding result of the first spatial slice of the nth receiver latent feature quantized representation channel slice and the decoding result corresponding to the encoding result of the second spatial slice of the nth receiver latent feature quantized representation channel slice constitute the decoding result of the nth receiver latent feature quantized representation channel slice (i.e. the decoded nth receiver latent feature quantized representation channel slice).

[0108] ​​​​​In one embodiment, based on the above embodiments, the semantic communication system is trained based on an initial semantic system. The initial semantic communication system includes an initial transmitter and an initial receiver. The initial transmitter includes at least an initial encoder, a first initial arithmetic encoder, a first initial synthesis transform module, a first initial entropy model, and a second initial arithmetic encoder. The initial receiver includes at least a second initial synthesis transform module, a second initial entropy model, and a first initial arithmetic decoder. During the initial semantic communication system process, the original sample image is input into the initial semantic communication system and processed by the initial transmitter and initial receiver until the output sample reconstructed image is obtained. Then, based on the original sample image and the sample reconstructed image, the model parameters of at least the initial encoder, the first initial arithmetic encoder, the first initial synthesis transform module, the first initial entropy model, the second initial arithmetic encoder, the second initial synthesis transform module, the second initial entropy model, and the first initial arithmetic decoder are updated until a trained transmitter and receiver are obtained, i.e., a trained semantic communication system is obtained, which serves as the semantic communication system in the above embodiments of the present invention.

[0109] In conjunction with the above embodiments, in one embodiment, it is assumed that the first... Spatial slice The encoded result is lost during transmission (i.e., the receiving end does not receive it). The encoding result), the receiver can directly represent the mean of the first spatial slice of the channel slice using the first receiver latent feature quantization. Replacement for missing The encoding result is used to complete the second spatial slice of the channel slice representing the latent feature quantization of the i-th receiver. Decoding the encoded result yields the decoded result. After obtaining the decoded... and decoded Then, these two decoding results are considered as decoded. slice That is, the decoded .

[0110] Assumption The encoding result is lost or The encoded result is lost (i.e., the receiver does not receive it). The encoding result or The encoded result is equivalent to the receiver not receiving the signal. The encoding result), the receiver can directly represent the mean of the first spatial slice of the channel slice using the first receiver latent feature quantization. Replacement for missing The encoding result is represented by the mean of the second spatial slice of the channel slice, quantized using the latent features of the first receiver. Replacement for missing The encoding result, thus obtaining The encoding result, thus completing the encoding of Decoding the encoded result yields the decoded result (i.e., the decoded version). ).

[0111] Assumption The encoded result is lost (i.e., the receiver does not receive it). (The encoding result), the receiver can directly represent the mean of the first spatial slice of the channel slice using the second receiver latent feature quantization. Replacement for missing The encoding result, thus completing the encoding of Decoding the encoded result yields The decoding result of the encoding result (i.e., the decoded result) ).

[0112] After obtaining the decoded and decoded And the decoded and decoded Then, these four decoding results are considered as decoded. slice That is, the decoded The decoding process for subsequent channel slices follows the same logic and will not be described in detail here.

[0113] Assuming out-of-order transmission occurs, the receiving end receives the data out of order. The processing procedure at the receiving end for each encoded result is as follows:

[0114] The receiving end first directly with Replacement for missing The encoding result, in Replacement for missing The encoding result is directly used as Replacement for missing The encoding result completes the encoding of the code. Decoding the encoded result. When After the encoded result is correctly received, then... The encoded result is placed in the corresponding position, and subsequent slices are decoded again. In this embodiment, the superscript represents the channel slice index, and the subscript represents the spatial slice index.

[0115] In one embodiment, such as Figure 5 As shown, Figure 5This is a schematic diagram illustrating the processing procedure of the transmitting end according to an embodiment of the present invention. For the input raw image x, the transmitting end first extracts the corresponding latent features y through an encoder. On the other hand, the latent features y are then segmented into multiple channel slices. Then Inputting the first quantization module Q1, we obtain N channel slices representing the latent features of the transmitter. (In this invention, channel segmentation can be performed before quantization, or quantization can be performed before channel segmentation; there is no limitation on this.) Furthermore, this embodiment also divides each channel slice into two parts according to a checkerboard pattern, denoted as... , , , , ..., , .

[0116] On the other hand, the latent feature y is input into the analysis and transformation module. The prior feature z is extracted and then processed by the second quantization module Q2 to obtain the quantized representation of the prior feature encoding at the transmitting end. On the one hand, the prior features encoded at the sending end are quantized and represented. The super-prior feature bitstream Z is obtained by encoding with the first arithmetic encoder AE1, and then transmitted to the receiving end. On the other hand, the transmitting side decodes the super-prior feature bitstream Z with the second arithmetic decoder AD2 to obtain the transmitting end's super-prior feature decoded quantization representation. Then, the transmitting end uses the first synthesis and transformation module. Get the sender context parameters , and, through the first synthesis transformation module Using sender context parameters The channel slices representing the latent features of the first transmitter are obtained respectively. corresponding variance The second transmitter latent feature quantization representation channel slice corresponding variance ..., the Nth transmitter latent feature quantization representation channel slice corresponding variance .

[0117] Then, the channel slice is represented by the quantization of the latent features of the first transmitter. Sending context parameters Inputting the first entropy model, the mean of the first spatial slice of the channel slice, representing the latent feature quantization of the first transmitter, is obtained through the channel module. Thus, through the second arithmetic encoder AE2 according to and the first spatial slice of the first latent feature quantized representation channel slice is encoded to obtain the encoding result of The encoding result of and the sender context parameter are input into the spatial module of the first entropy model to obtain the mean value of the second spatial slice of the first latent feature quantized representation channel slice , so that the second arithmetic encoder AE2 encodes the second spatial slice of the first latent feature quantized representation channel slice and to obtain the encoding result of After obtaining the encoding result of and the encoding result of , the encoding results of the two are taken as the encoding results of the encoded slices .

[0118] For the second latent feature quantized representation channel slice , the encoding result of (the encoding result of and the encoding result of ) and the sender context parameter are input into the first entropy model to obtain the mean value of the first spatial slice of the second latent feature quantized representation channel slice through the channel module, so that the second arithmetic encoder AE2 encodes the first spatial slice of the second latent feature quantized representation channel slice and to obtain the encoding result of The encoding result of and the sender context parameter are input into the first entropy model to obtain the mean value of the second spatial slice of the second latent feature quantized representation channel slice through the spatial module, so that the second arithmetic encoder AE2 encodes the second spatial slice of the second latent feature quantized representation channel slice and to obtain the encoding result of After obtaining the encoding result of and the encoding result of , and the encoding result of and the encoding result of , the encoding results of the two are taken as the encoding results of the encoded slices ​the encoding results of the four are taken as the encoded slices , and so on, to obtain the encoding results of and the encoding results of . Among them, the encoding results of and the encoding results of constitute a latent feature bit stream 1, and the encoding results of constitute a latent feature bit stream 2, …, and the encoding results of constitute a latent feature bit stream N, and the latent feature bit streams are sent to the receiving end. In the embodiment, the superscript represents the channel slice index, and the subscript represents the spatial slice index.

[0119] In an embodiment, as shown in Figure 6 , the receiving end processing process shown in an embodiment of the application is shown in Figure 6 . The receiving end can decode the received hyper-prior feature bit stream z through a third arithmetic decoder AD3 to obtain a receiving end hyper-prior feature decoded quantization representation , and then process the receiving end hyper-prior feature decoded quantization representation through a second synthesis transformation module to obtain receiving end context parameters . Similar to the sending end, the receiving end utilizes the receiving end context parameters through the second synthesis transformation module to respectively obtain a variance corresponding to the latent feature bit stream 1, a variance corresponding to the latent feature bit stream 2, …, and a variance corresponding to the latent feature bit stream N.

[0120] Then, the receiving end context parameters are input into a second entropy model for the latent feature bit stream 1 (i.e., corresponding to the first receiving end latent feature quantization representation channel slice), to obtain a mean value of the first spatial slice of the first receiving end latent feature quantization representation channel slice through a channel module. In this way, the encoding results of the first spatial slice of the first receiving end latent feature quantization representation channel slice (i.e., the encoding results of the first spatial slice in the latent feature bit stream 1) are decoded through a first arithmetic decoder AD1 according to and to obtain the first spatial slice of the first receiving end latent feature quantization representation channel slice decoded slice ). The and the receiver context parameter are input into the spatial module of the second entropy model to obtain the mean of the second spatial slice of the first receiver latent feature quantized representation channel slice . In this way, the encoding result (i.e., the encoding result of the second spatial slice in the latent feature bitstream 1) of the second spatial slice of the first receiver latent feature quantized representation channel slice is decoded by the first arithmetic decoder AD1 according to , and the decoding result (i.e., the decoding result of the second spatial slice of the first receiver latent feature quantized representation channel slice ) of the encoding result of the second spatial slice of the first receiver latent feature quantized representation channel slice is obtained. After obtaining the decoded and , the decoding results of the two are taken as the decoded slices , i.e. . .

[0121] For the latent feature bitstream 2 (i.e., corresponding to the second receiver latent feature quantized representation channel slice), the decoded and the receiver context parameter are input into the second entropy model, and the mean of the first spatial slice of the second receiver latent feature quantized representation channel slice is obtained by the channel module. In this way, the encoding result (i.e., the encoding result of the first spatial slice in the latent feature bitstream 2) of the first spatial slice of the second receiver latent feature quantized representation channel slice is decoded by the first arithmetic decoder AD1 according to , and the decoding result (i.e., the decoding result of the first spatial slice of the second receiver latent feature quantized representation channel slice ) of the encoding result of the first spatial slice of the second receiver latent feature quantized representation channel slice is obtained. The and the receiver context parameter are input into the spatial module of the second entropy model to obtain the mean of the second spatial slice of the second receiver latent feature quantized representation channel slice . In this way, the encoding result (i.e., the encoding result of the second spatial slice in the latent feature bitstream 2) of the second spatial slice of the second receiver latent feature quantized representation channel slice is decoded by the first arithmetic decoder AD1 according to and , and the decoding result (i.e., the decoding result of the second spatial slice of the second receiver latent feature quantized representation channel slice The encoding result (i.e., the encoding result of the second spatial slice in the latent feature bitstream 2) is decoded to obtain the second spatial slice of the second receiver latent feature quantization representation channel slice. The decoding result of the encoding result (i.e., the slice obtained by decoding) After obtaining the decoded result... and Then, the decoding results of both are taken as the decoded data. slice After obtaining the decoded... and And the decoded and Then, the decoding results of these four are used as the decoded slices. And so on, until the decoded result is obtained. and Among them, the decoded and As decoded slice The receiving end can obtain the decoded data. , ,……, By splicing the pieces together, we obtain spliced ​​slices. Then, based on the decoding end, the spliced ​​slices are... Perform semantic decoding to obtain the reconstructed image. In this embodiment, the superscript represents the channel slice index, and the subscript represents the spatial slice index.

[0122] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0123] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the program implements the steps of the deep learning progressive out-of-order transmission image compression method for satellite communication described in any of the above embodiments of the present invention.

[0124] The various embodiments described in this specification are intended to be illustrative only and in no way limit the scope of the application. One skilled in the art will readily recognize from the disclosure herein, possible alternative techniques within the scope of the application. Accordingly, the embodiments described in this specification are set forth without any loss of generality to, and without imposing limitations upon, the claimed invention.

[0125] Those skilled in the art will appreciate that embodiments of the application can be devised for a variety of applications. It is therefore intended that the appended claims be construed to include alternative embodiments to the extent permitted.

[0126] Embodiments of the present application are described herein with reference to the drawings, in which are shown Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing each function specified in a flow or multiple flows and / or blocks.

[0127] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing each function specified in a flow or multiple flows and / or blocks.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more functions specified in a flow or multiple flows and / or blocks. Figure 1 means for performing each function specified in a flow or multiple flows and / or blocks.

[0129] While the preferred embodiments of the application have been described above, it will be recognized and understood that various modifications and changes can be made to these embodiments by those skilled in the art having the benefit of this detailed description without departing from the spirit and scope of the application. It is therefore intended that the appended claims cover all such modifications and changes as fall within the scope of the application.

[0130] Finally, it is to be understood that the phraseology or terminology employed herein, such as "first" and "second", etc., are for descriptive purposes only and should not be construed to connote or otherwise imply any kind of ordering, precedence or relationships between or among the elements or operations so described. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0131] The above describes in detail a satellite communication oriented deep learning progressive out-of-order transmission image compression method and device provided by the present application, and the principles and implementation manners of the present application are described by using specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed, and the above description of the present application should not be understood as a limitation of the present application.

Claims

1. A deep learning-based progressive out-of-order transmission image compression method for satellite communication, characterized in that, The method includes: In a semantic communication system, the transmitter extracts latent features from the original image through an encoder to obtain the transmitter's latent feature quantization representation and the corresponding transmitter's prior feature encoding quantization representation and transmitter's prior feature decoding quantization representation. The transmitting end encodes the quantized representation of the transmitting end's prior feature code using a first arithmetic encoder to obtain a prior feature bit stream, and then sends the prior feature bit stream to the receiving end in the semantic communication system. The transmitting end processes the decoding and quantization representation of the transmitting end's prior features through the first synthesis and transformation module to obtain the transmitting end's context parameters, and generates the variance of the transmitting end's latent feature quantization representation based on the transmitting end's context parameters; The transmitting end obtains the mean value of the quantized representation of the latent features of the transmitting end at least based on the transmitting end context parameters through the first entropy model; The transmitting end encodes the latent feature quantization representation of the transmitting end using a second arithmetic encoder based on the variance and mean of the latent feature quantization representation of the transmitting end, thereby obtaining a latent feature bit stream, and then sends the latent feature bit stream to the receiving end.

2. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 1, characterized in that, The method further includes: The receiving end processes the receiver's prior feature decoding quantization representation through the second synthesis transformation module to obtain the receiver's context parameters, and generates the variance of the receiver's latent feature quantization representation corresponding to the latent feature bitstream received by the receiving end based on the receiver's context parameters; the receiver's prior feature decoding quantization representation is obtained by the receiving end decoding the received prior feature bitstream. The receiving end obtains the mean value of the quantized representation of the latent features of the receiving end at least based on the receiving end context parameters through the second entropy model; The receiving end decodes the received latent feature bitstream using a first arithmetic decoder based on the variance and mean of the latent feature quantization representation of the receiving end, so as to obtain the reconstructed image.

3. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 2, characterized in that, The latent feature bitstream includes the encoding results of the quantized representation channel slices of the latent features from the first to the Nth transmitter; the method further includes: If the receiving end does not receive the encoding result of the nth transmitting end potential feature quantization representation channel slice in the potential feature bit stream, it takes the average value of the nth receiving end potential feature quantization representation channel slice as the encoding result of the received nth receiving end potential feature quantization representation channel slice. The receiver obtains the mean value of the quantized representation of the latent features of the receiver at least based on the receiver context parameters through a second entropy model, including: The receiving end obtains the mean value of the nth receiver latent feature quantization representation channel slice by means of the second entropy model based on the receiving end context parameters and the encoding results of the received 1st to nth (n-1)th receiver latent feature quantization representation channel slices respectively.

4. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 3, characterized in that, After using the mean of the nth receiver latent feature quantization representation channel slice as the encoding result of the received nth receiver latent feature quantization representation channel slice, the method further includes: When the receiving end receives the encoding result of the nth latent feature quantization representation channel slice in the latent feature bit stream, the receiving end, based on the receiving end context parameters and the encoding results of the 1st to nth latent feature quantization representation channel slices respectively, re-obtains the mean value of the (n+1)th latent feature quantization representation channel slice through the second entropy model. The receiving end uses the variance of the channel slice represented by the latent feature quantization of the (n+1)th receiving end and the re-obtained mean to re-decode the encoding result of the channel slice represented by the latent feature quantization of the (n+1)th receiving end through the first arithmetic decoder to obtain the updated reconstructed image.

5. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 1, characterized in that, In a semantic communication system, the transmitter extracts latent features from the original image through an encoder to obtain a quantized representation of the latent features and corresponding quantized and decoded representations of the prior features, including: The transmitting end maps the latent features of the original image to the quantized representation of the latent features of the transmitting end through the first quantization module; The transmitting end extracts the prior features of the original image from the latent features of the original image through the analysis and transformation module, and maps the prior features of the original image to the prior feature encoding and quantization representation of the transmitting end through the second quantization module; The sending end stores the priori feature bit stream locally; The transmitting end decodes the super-prior feature bitstream using a second arithmetic decoder to obtain the super-prior feature decoded quantization representation of the transmitting end.

6. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 5, characterized in that, After the transmitting end obtains the quantized representation of the latent features, the method further includes: The transmitting end performs channel segmentation on the latent feature quantization representation of the transmitting end to obtain N channel slices of the latent feature quantization representation of the transmitting end; The variance of the latent feature quantization representation generated based on the sender context parameters includes: The transmitter generates the variance of the channel slice by the nth latent feature quantization representation based on the transmitter context parameters, where n is an integer from 1 to N. The transmitting end encodes the latent feature quantization representation based on the variance and mean of the latent feature quantization representation using a second arithmetic encoder to obtain a latent feature bitstream, including: The transmitting end encodes the nth latent feature quantization representation channel slice based on the variance and mean of the nth latent feature quantization representation channel slice using the second arithmetic encoder to obtain the encoding result of the nth latent feature quantization representation channel slice. The latent feature bitstream consists of the encoding results of the channel slices represented by the quantization of the first to Nth latent features at the transmitting end.

7. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 6, characterized in that, The transmitting end obtains the mean value of the quantized representation of the latent features of the transmitting end through a first entropy model, based at least on the transmitting end context parameters, including: The transmitting end obtains the mean value of the first transmitting end potential feature quantization representation channel slice through the first entropy model based on the transmitting end context parameters. The transmitting end obtains the mean value of the quantized representation channel slice of the nth transmitter latent feature through the first entropy model based on the transmitting end context parameters and the encoding results of the 1st to n-1th transmitter latent feature quantization channel slices. The value of n is an integer from 2 to N.

8. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 7, characterized in that, After obtaining N channel slices representing the latent features of the transmitters, the method further includes: Following a checkerboard pattern, the transmitting end spatially partitions the nth latent feature quantization representation channel slice to obtain the nth latent feature quantization representation channel slice corresponding to the nth latent feature quantization representation channel slice. Spatial slice and the first Spatial slice, where n is an integer from 1 to N; The transmitting end obtains the mean value of the first transmitting end latent feature quantization representation channel slice through the first entropy model based on the transmitting end context parameters, including: The transmitting end obtains the first entropy model through the channel module in the first entropy model based on the transmitting end context parameters. The mean of the spatial slice; the transmitter, based on the transmitter context parameters and the first... The encoding result of the spatial slice, through the spatial module in the first entropy model, yields the first... Mean of a spatial slice; The transmitting end, based on the transmitting end context parameters and the encoding results of the 1st to (n-1st)th transmitting end latent feature quantization representation channel slices, obtains the mean value of the nth transmitting end latent feature quantization representation channel slice through the first entropy model, including: The transmitting end quantizes the encoding results of each channel slice according to the transmitting end context parameters and the 1st to n-1th latent features of the transmitting end, and obtains the encoding result of the channel slice through the channel module in the first entropy model. Mean of a spatial slice; The sending end, based on the sending end context parameters and the first... The encoding result of the spatial slice, through the spatial module in the first entropy model, yields the first... Mean of spatial slices.

9. The deep learning-based progressive out-of-order transmission image compression method for satellite communication according to claim 8, characterized in that, The transmitting end encodes the nth latent feature quantization representation channel slice based on the variance and mean of the nth latent feature quantization representation channel slice using the second arithmetic encoder, obtaining the encoding result of the nth latent feature quantization representation channel slice, including: The sending end is based on the first The mean of the spatial slice and the variance of the channel slice are represented by the latent feature quantization of the nth transmitter. This variance is then processed by the second arithmetic encoder. Encoding spatial slices yields the first... Encoding results of spatial slices; The sending end is based on the first The mean of the spatial slice and the variance of the channel slice are represented by the latent feature quantization of the nth transmitter. This variance is then processed by the second arithmetic encoder. Encoding spatial slices yields the first... The encoding result of the spatial slice, the first The encoding results of spatial slices and the first The encoding results of the spatial slice constitute the encoding results of the channel slice of the nth transmitter latent feature quantization representation.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the deep learning progressive out-of-order transmission image compression method for satellite communication as described in any one of claims 1 to 9.

Images