Image transmission method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image transmission, and provides an image transmission method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting a to-be-transmitted image into an image transmission model, and obtaining a target image; the image transmission model comprises an encoder, a quantization part, a pseudo decoder and a decoder; the encoder comprises a plurality of encoding modules connected in series, and the pseudo decoder comprises a plurality of pseudo decoding modules; quantization strategies of the quantization part correspond to different noise channels; the encoder performs image encoding on the to-be-transmitted image based on each encoding module to obtain each encoding feature; the pseudo decoder decodes the codeword features corresponding to the quantization modules based on the pseudo decoding modules to obtain intermediate features; the quantization part obtains a target feature based on each coding feature and each intermediate feature; and the decoder decodes the target feature to obtain a target image. In the quantization part, the quantization strategy is flexibly adjusted according to the noise characteristic of each channel, the transmission potential of each channel is fully exerted, and the overall transmission efficiency and quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of image transmission technology, and in particular to an image transmission method, device, electronic equipment and storage medium. Background Art

[0002] Research, finance, IT, and manufacturing institutions process massive amounts of data daily. This data is crucial for intelligent production, refined management, market responsiveness, production process optimization, and enhanced competitiveness. However, fast and secure data transmission faces challenges such as channel noise, bandwidth bottlenecks, latency, and security. Effective data encoding algorithms can reduce data size, improve transmission efficiency and resource utilization, and ensure data security.

[0003] Existing technologies primarily face channel capacity limitations and channel noise adaptability issues. The channel capacity limitation refers to the fact that current transmission methods often send all data at once through a single channel, making it difficult to effectively handle the transmission needs of multiple smaller channels. For example, after conventional encoding, the data size of an image or a batch of images may reach 100MB. However, there are only five available channels, with maximum transmission capacities of 1MB, 3MB, 5MB, 7MB, and 9MB per channel, respectively. In this case, the image data cannot be transmitted all at once through a single channel. The image must be segmented or sent multiple times, which reduces the overall image transmission quality. The channel noise adaptability issue refers to the fact that noise levels vary significantly across channels. Some channels are severely noisy, resulting in poor transmission quality, while others are less noisy and have relatively good transmission quality. However, existing transmission schemes typically use the same coding method for all channels, failing to flexibly adjust the coding strategy based on the noise characteristics of each channel. This prevents the full utilization of the transmission potential of each channel, further impacting overall transmission efficiency and quality. Summary of the Invention

[0004] The present invention provides an image transmission method, device, electronic device and storage medium, which are used to solve the defects of channel capacity limitation and channel noise adaptability in the prior art.

[0005] The present invention provides an image transmission method, comprising the following steps: Obtaining the image to be transmitted; Inputting the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0006] According to an image transmission method provided by the present invention, the quantization part includes a plurality of quantization modules, and the quantization strategies of the plurality of quantization modules correspond to different noise channels; For any one of the plurality of serially connected encoding modules, a corresponding quantization module is configured; The quantization modules are numbered from 1 to N, and the pseudo-decoding modules are numbered from 2 to N, wherein, for any value of i from N-1 to 1, the i-th quantization module corresponds to the i+1-th pseudo-decoding module; When i is equal to N, the i-th quantization module is used to quantize and encode the coding features of the i-th coding module to obtain the top-level codeword features; the i-th pseudo-decoding module is used to decode the top-level codeword features corresponding to the i-th quantization module to obtain the top-level intermediate features; For the coding feature output by the i-th encoder, when i is not equal to N, the i-th encoding module performs image encoding on the image to be transmitted to obtain the i-th coding feature, and the i-th quantization module splices the intermediate feature of the i+1-th pseudo decoding module with the i-th coding feature to obtain the i-th codeword feature; Iteratively perform steps from i=N-1 to i=1 to obtain codeword features corresponding to multiple quantization modules, and determine the target feature based on the codeword features corresponding to the multiple quantization modules.

[0007] According to an image transmission method provided by the present invention, determining the target feature based on the codeword features corresponding to the multiple quantization modules includes: Concatenate the i-th codeword feature corresponding to the i-th quantization module and the i-1-th codeword feature corresponding to the i-1-th quantization module to obtain the first codeword feature; Concatenate the first codeword feature with the (i-2)th codeword feature corresponding to the (i-2)th quantization module to obtain a second codeword feature; Traverse from the code word feature corresponding to the Nth quantization module to the code word feature corresponding to the 1st quantization module until the target feature is obtained.

[0008] According to an image transmission method provided by the present invention, the training step of the image transmission model includes: Obtaining an initial image transmission model and a sample transmission image; Inputting the sample transmission image into the initial image transmission model to obtain a predicted target image output by the initial image transmission model; Based on the difference between the predicted target image and the sample transmission image, a target loss is determined, and parameters of the initial image transmission model are iterated based on the target loss to obtain the image transmission model.

[0009] According to an image transmission method provided by the present invention, determining the target loss based on the difference between the predicted target image and the sample transmission image includes: Obtaining an initial quantization module, sample coding features, and an initial dictionary set of the sample coding features; Based on the initial quantization module, feature mapping is performed on the sample coding feature and the codewords in the initial dictionary set to obtain the quantized codeword feature output by the initial quantization module; determining an internal loss based on a difference between the quantized codeword feature and the sample coding feature; determining a transmission loss based on a difference between the predicted target image and the sample transmission image; The target loss is determined based on the internal loss and the transmission loss.

[0010] According to an image transmission method provided by the present invention, the initial quantization module is used to perform feature mapping on the sample coding features and the codewords in the initial dictionary set to obtain the quantization codeword features output by the initial quantization module, including: Based on the initial quantization module, a Euclidean distance between the sample coding feature and the codewords in the initial dictionary set is determined, a target codeword is determined based on the Euclidean distance, and a quantized codeword feature output by the initial quantization module is determined based on the target codeword.

[0011] According to an image transmission method provided by the present invention, the encoder includes a convolutional layer, a residual layer and an activation layer in sequence; The pseudo decoder includes a convolution layer, a residual layer, a deconvolution layer and an activation layer in sequence.

[0012] According to an image transmission method provided by the present invention, the decoder includes a convolution layer, a residual layer, an activation layer, a deconvolution layer and a deconvolution layer in sequence.

[0013] The present invention also provides an image transmission device, comprising the following units: An acquisition unit, configured to acquire an image to be transmitted; A transmission unit, configured to input the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-described image transmission methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned image transmission methods when executed by a processor.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned image transmission methods.

[0017] The image transmission method, apparatus, electronic device, and storage medium provided by the present invention acquire an image to be transmitted, then input the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model. The image transmission model includes an encoder, a quantization unit, a pseudo-decoder, and a decoder. The encoder includes multiple serially connected encoding modules, and the pseudo-decoder includes multiple pseudo-decoding modules. The quantization unit's quantization strategy corresponds to different noise channels. The encoder is configured to encode the image to be transmitted based on each encoding module to obtain encoding features. The pseudo-decoder is configured to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain intermediate features. The quantization unit is configured to obtain target features based on each encoding feature and each intermediate feature. The decoder is configured to decode the target feature to obtain a target image. The quantization unit generates target features based on each encoding feature and each intermediate feature. Based on this, the quantization strategy is flexibly adjusted according to the noise characteristics of each channel. While meeting channel capacity requirements, the optimal subsequent decoding effect is ensured as much as possible, thereby fully utilizing the transmission potential of each channel and improving overall transmission efficiency and quality. Furthermore, because the quantization strategy can be optimized for different noise channels, there is no need to segment or transmit the image multiple times, thereby improving the overall quality of image transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the image transmission method provided by the present invention.

[0020] Figure 2 This is the second flow chart of the image transmission method provided by the present invention.

[0021] Figure 3 It is a schematic diagram of a receiving end in the image transmission method provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the sending end of the two-layer semantic information extraction, quantization and transmission method provided by the present invention.

[0023] Figure 5 It is a schematic diagram of the receiving end of the two-layer semantic information extraction, quantization and transmission method provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the image transmission device provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] Among related technologies, 6G networks and semantic communication are key to future data transmission. 6G is expected to offer higher transmission rates, lower latency, and wider coverage than 5G, supporting more efficient data encoding and transmission. Semantic communication focuses on the meaning and purpose of data, rather than simply transmitting the data itself. Combined with efficient encoding techniques, this can further improve transmission speed and security, facilitating digital transformation.

[0028] The core challenge of image data transmission is efficiently compressing and transmitting large amounts of image data while maintaining image quality. Traditional methods such as JPEG and JPEG2000 encode images using the Discrete Cosine Transform (DCT) or the Discrete Wavelet Transform (DWT), exploiting the human eye's insensitivity to high-frequency information to perform quantization and reduce data size. Modern coding standards such as H.265 / HEVC introduce complex quantization matrices and entropy coding techniques to further improve compression efficiency. Channel noise and bandwidth limitations are major challenges in wireless image transmission. Channel coding techniques such as convolutional coding, Turbo coding, and LDPC coding enhance noise immunity by introducing redundant data.

[0029] In recent years, deep learning technology has made significant progress in image processing. Codecs based on deep neural networks achieve efficient compression and high-quality reconstruction by learning nonlinear transformations of images. For example, convolutional neural networks (CNNs) extract semantic features from images, compress data through quantization and entropy coding, and reconstruct the image using deconvolution and upsampling techniques. Deep learning-driven image compression algorithms fully exploit data semantics, making them ideal for data transmission in the 6G environment.

[0030] However, these existing methods suffer from channel capacity limitations and channel noise adaptability issues. The first is that image data cannot be transmitted all at once through a single channel, requiring the image to be segmented or sent multiple times, which reduces overall image transmission quality. The second is that the same coding method is typically used for all channels, preventing the coding strategy from being flexibly adjusted based on the noise characteristics of each channel. Consequently, the transmission potential of each channel cannot be fully utilized, further impacting overall transmission efficiency and quality.

[0031] Based on the above problems, the present invention provides an image transmission method. Figure 1 This is one of the flow charts of the image transmission method provided by the present invention. Figure 2 This is the second flow chart of the image transmission method provided by the present invention, such as Figure 1 、 Figure 2 As shown, the method includes step 110 and step 120.

[0032] Step 110: Acquire the image to be transmitted.

[0033] Specifically, an image to be transmitted can be obtained, that is, an image that needs to be transmitted subsequently. The image to be transmitted can be pre-captured by an image acquisition device, or can be captured in real time, or can be downloaded or scanned from the Internet. The embodiment of the present invention does not specifically limit this.

[0034] Step 120: input the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0035] Specifically, after the image to be transmitted is acquired, the image to be transmitted may be input into the image transmission model to obtain a target image output by the image transmission model.

[0036] Consider the case of large-scale image transmission with multiple low-bandwidth noisy channels. The image to be transmitted can be represented by x, with a size of C (channel) × H (height) × W (weight). Suppose N layers of semantic information are extracted. The entire processing process is as follows: Figure 2 As shown, the image transmission model includes an encoder, a quantization part, a pseudo decoder, and a decoder. The encoder includes multiple serially connected encoding modules, such as Enc1, Enc2, ..., EncN, and the pseudo decoder includes multiple pseudo decoding modules, such as Dec'2, Dec'3, ..., Dec'N. The quantization strategies of the quantization part correspond to different noise channels.

[0037] The encoder encodes the image to be transmitted based on each encoding module to obtain encoding features. The pseudo-decoder decodes the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain intermediate features. The quantization module obtains target features based on the encoding features and intermediate features. The decoder decodes the target features to obtain the target image.

[0038] It should be noted that the image transmission model is set between the sending end and the receiving end. The sending end is distributed with an encoder, a quantization part and a pseudo decoder, and the receiving end is distributed with a decoder.

[0039] The encoder consists of a convolutional layer, a residual layer, and an activation layer in sequence, while the pseudo decoder consists of a convolutional layer, a residual layer, a deconvolutional layer, and an activation layer in sequence. The decoder consists of a convolutional layer, a residual layer, an activation layer, a deconvolutional layer, and a deconvolutional layer.

[0040] It should be noted that the convolutional layer can use a multi-layer convolutional neural network (CNN) with a cascade structure, or a deep neural network (DNN), or a combination of CNN and DNN, etc., which is not specifically limited in the embodiment of the present invention.

[0041] The activation layers here may use a GELU (Gaussian Error Linear Unit) activation function, a Sigmoid activation function, or a ReLU (Rectified Linear Units) activation function, which is not specifically limited in the embodiment of the present invention.

[0042] In a preferred embodiment, the present invention defines an encoder consisting of 2-3 convolutional layers, 1 residual layer, and 1 additional activation function, arranged in sequence. By default, all activation functions are ReLU. Each convolutional layer includes a convolutional network and an activation function, where the convolutional network is used to extract image features. In an encoder containing three convolutional layers, the convolutional networks of the first two layers downsample the input data to 1 / 2 of its original resolution, meaning the extracted features are half the size of the input data. The convolutional network of the last layer does not change the resolution but only extracts features. The extracted features are the same size as the input image to be transmitted. The mathematical representation of the encoder is y = ReLU(Conv(ReLU(Conv(ReLU(Conv(x)))))), where x is the input, i.e., the image to be transmitted, with a shape of c×h×w. The output y has a shape of c×h / 4×w / 4. In an encoder consisting of two convolutional layers, the convolutional network in the first convolutional layer compresses the original resolution to 50% through downsampling, extracting features that are half the size of the input data. The convolutional network in the second convolutional layer only extracts data features without changing the resolution. That is, y = ReLU(Conv(ReLU(Conv(x)))), where x is the input and has a shape of c × h × w, then the output y has a shape of c × h / 2 × w / 2. Through this layered encoder design, the model can gradually extract features of the input data and provide rich information for subsequent quantization and decoding.

[0043] The residual layer consists of two activation functions and two convolutional networks arranged in an interlaced manner, followed by a final add-back operation. The convolutional network feature extraction does not change the resolution of the input data, i.e., y = f(x) + x, where f(x) = Conv(ReLU(Conv(ReLU(x)))) , and the output y has the same shape as the input x. The residual block first directly adds the input x to the output f(x), allowing gradients to propagate directly through x, alleviating the vanishing gradient problem in deep networks. Furthermore, the residual block learns more complex features, allowing the network to learn the residual f(x) = yx between the input and output, rather than directly learning the output y. This makes the network easier to optimize and capable of learning more complex features. Finally, the output of the residual block has the same resolution as the input, making it easy to combine with other layers (such as convolutional or pooling layers) without changing the size of the feature map.

[0044] It's understandable that residual layers alleviate the vanishing gradient problem in deep networks by directly adding the input to the output, making the network easier to optimize and capable of learning more complex features. Furthermore, the output of the residual layer has the same resolution as the input, making it easier to combine with other layers.

[0045] The image to be transmitted is first encoded by N encoders. Because the number of input channels, number of output channels, and feature extraction density of the encoders vary for each encoding operation, for ease of understanding, this document considers encoders with different parameter settings and accepting different input data as different encoders, denoted as Enc1 to EncN. However, the underlying logic and basic operating operations of each encoder are the same.

[0046] The encoders for N encodings are Enc1, Enc2, ..., EncN, where Enc1 has three convolutional layers, and its input is the image to be transmitted (the image to be transmitted). Each subsequent encoder has only two convolutional layers, and the input is the output of the previous encoder. The result after each encoding is char1, char2, ..., charN, which represent the N layers of features from the bottom layer to the top layer of the image to be transmitted. As the more encoders the data passes through, the more abstract and general the semantic feature information represented by the encoding result, the more obvious the compression effect, and the lower the resolution. Therefore, the shape of Enc1's output char1 is 1 / 4 of the original image. 2 , each subsequent output char is 1 / 2 of the previous char 2 .

[0047] It should be noted that the quantization module does not change the size of the data. If the input x shape is c in ×h×w, then the shape of the quantized codeword is also c out×h×w. This discretization process enables the model to learn more compact representations and can be used for unsupervised learning, compression, and generation tasks. In this process, input features are mapped to a predefined discrete set (codeword set), thereby discretizing the features. This discretization not only helps reduce model complexity but also improves model generalization capabilities.

[0048] It can be understood that the decoder gradually improves the resolution of the feature codeword through a combination of convolutional networks, residual layers, and deconvolution layers, and finally restores it to a resolution consistent with the original input data (the image to be transmitted), ensuring the quality of image restoration.

[0049] In the embodiment of the present invention, when quantizing codewords, the high-level features are used as conditions, and are combined with the low-level features to input into the quantization module to obtain the bottom-level quantized codewords. However, since the feature char of the higher level is 1 / 2 smaller than the char of the lower level, 2 . The function of the pseudo decoder is to eliminate the size difference between the two layers of char, so that the high-level char can be stretched to the same size as the low-level char, and then quantized into codewords after splicing. All pseudo decoders are composed of 1 convolutional layer, 1 residual layer, 1 deconvolution layer and 1 additional activation function connected in series, where all activation functions are ReLU. The definitions of the convolutional layer and the residual layer are the same as the encoder. The convolutional network in the convolutional layer only extracts the semantic features of the input data without changing the resolution. The deconvolution layer consists of a deconvolution network and an activation function. The deconvolution network uses upsampling to increase the resolution of the high-level char by 2 times, making it the same size as the low-level char, that is, y=ReLU(ConvTranspose2d (x)), x is the input, and the shape is c×h×w, then the output y has a shape of c×2h×2w.

[0050] The method provided by an embodiment of the present invention obtains an image to be transmitted, then inputs the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model. The image transmission model includes an encoder, a quantization unit, a pseudo-decoder, and a decoder. The encoder includes multiple serially connected encoding modules, and the pseudo-decoder includes multiple pseudo-decoding modules. The quantization unit's quantization strategy corresponds to different noise channels. The encoder encodes the image to be transmitted based on each encoding module to obtain encoding features. The pseudo-decoder decodes the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain intermediate features. The quantization unit obtains target features based on the encoding features and intermediate features. The decoder decodes the target features to obtain a target image. The quantization unit generates target features based on the encoding features and intermediate features. Based on this, the quantization strategy is flexibly adjusted according to the noise characteristics of each channel. While meeting channel capacity requirements, the optimal subsequent decoding effect is ensured as much as possible, thereby fully utilizing the transmission potential of each channel and improving overall transmission efficiency and quality. Furthermore, because the quantization strategy can be optimized for different noise channels, there is no need to segment or transmit the image multiple times, thereby improving the overall quality of image transmission.

[0051] Based on the above embodiment, the quantization part includes multiple quantization modules, and the quantization strategies of the multiple quantization modules correspond to different noise channels; For any one of the plurality of serially connected encoding modules, a corresponding quantization module is configured; The quantization modules are numbered from 1 to N, and the pseudo-decoding modules are numbered from 2 to N, wherein, for any value of i from N-1 to 1, the i-th quantization module corresponds to the i+1-th pseudo-decoding module; When i is equal to N, the i-th quantization module is used to quantize and encode the coding features of the i-th coding module to obtain the top-level codeword features; the i-th pseudo-decoding module is used to decode the top-level codeword features corresponding to the i-th quantization module to obtain the top-level intermediate features; For the coding feature output by the i-th encoder, when i is not equal to N, the i-th encoding module performs image encoding on the image to be transmitted to obtain the i-th coding feature, and the i-th quantization module splices the intermediate feature of the i+1-th pseudo decoding module with the i-th coding feature to obtain the i-th codeword feature; Iteratively perform steps from i=N-1 to i=1 to obtain codeword features corresponding to multiple quantization modules, and determine the target feature based on the codeword features corresponding to the multiple quantization modules.

[0052] Specifically, the quantization part includes multiple quantization modules, and the quantization strategies of the multiple quantization modules correspond to different noise channels. For example, the quantization strategy of quantization module 1 corresponds to noise channel 1, the quantization strategy of quantization module 2 corresponds to noise channel 2, and the quantization strategy of quantization module N corresponds to noise channel N.

[0053] For any one of the multiple serially connected encoding modules, a corresponding quantization module is configured. For example, encoding module Enc1 corresponds to quantization module 1, encoding module Enc2 corresponds to quantization module 2, ..., encoding module EncN corresponds to quantization module N.

[0054] Among them, the serial numbers of the quantization modules are 1 to N, and the serial numbers of the pseudo-decoding modules are 2 to N. Among them, for any value of i from N-1 to 1, the i-th quantization module corresponds to the i+1-th pseudo-decoding module. For example, the quantization module with serial number N-1 corresponds to the pseudo-decoding module with serial number N, that is, the quantization module with serial number N-1 corresponds to the pseudo-encoder Dec'N,..., the quantization module with serial number 2 corresponds to the pseudo-encoder Dec'3, and the quantization module with serial number 1 corresponds to the pseudo-encoder Dec'2.

[0055] When i equals N, the i-th quantization module quantizes and encodes the encoded features of the i-th encoding module to obtain the top-level codeword features. The i-th pseudo-decoding module (pseudo-encoder Dec'N) decodes the top-level codeword features corresponding to the i-th quantization module to obtain the top-level intermediate features. The top-level intermediate features are input to quantization module N-1, which concatenates the top-level intermediate features output by pseudo-encoder Dec'N with encoder EncN-1 to obtain the codeword features corresponding to quantization module N-1.

[0056] For the coding features output by the i-th encoder, when i is not equal to N, the i-th encoding module performs image encoding on the image to be transmitted to obtain the i-th coding feature, and the i-th quantization module splices the intermediate features of the i+1-th pseudo-decoding module with the i-th coding feature to obtain the i-th codeword feature.

[0057] Iteratively execute steps from i=N-1 to i=1 to obtain codeword features corresponding to multiple quantization modules, and determine target features based on the codeword features corresponding to the multiple quantization modules.

[0058] To summarize, after the N encoders generate N layers of semantic features char1,…charN of the image, the top-level feature charN is first input into quantization module N, where it is quantized to obtain the top-level codeword, denoted as codeN. codeN is both sent through noisy channel N and input into the pseudo-decoder Dec'N. After being reshaped, it is concatenated with the output charN-1 of the lower-level EncN-1 encoder according to the channel dimension. It is then input into quantization module N-1 to obtain codeN-1. Similarly, codeN-1 is both sent through noisy channel N-1 and input into the pseudo-decoder Dec'N-1 for the next step. The final step is to concatenate the outputs of char1 and Dec'2 generated by the Enc1 encoder, input into quantization module 1 to obtain code1, and then send it to the receiver through noisy channel 1.

[0059] Since the shape of code3 is 1 / 2 of code2 2 times, the shape of code2 is 1 / 2 of code1 2 times, and so on. Therefore, when selecting a noisy channel, appropriate codewords can be selected for transmission based on the bandwidth of each channel. The smaller the channel bandwidth, the higher-level codewords are recommended for transmission. Conversely, the larger the channel bandwidth, the lower-level codewords are recommended for transmission. At this point, all quantized codewords, code1, ..., codeN, have been generated and sent through different noisy channels, and all processing on the transmitter is complete.

[0060] It should be noted that the embodiment of the present invention considers the Additive White Gaussian Noise Channel (AWGN) channel. Through a function The channel output symbol received by the source-channel joint decoder is Expressed as:

[0061] Among them, the vector It consists of independent and identically distributed (iid) samples, each sample follows Gaussian distribution, is the noise power.

[0062] The method provided in the embodiment of the present invention mines semantic information at different levels of an image through the design of multi-layer semantic information extraction, quantized transmission, and multi-layer information fusion decoding, thereby achieving goals such as efficient compression and transmission, improved channel utilization, adaptation to various channel conditions, optimized network performance, and high-quality image restoration.

[0063] 1. Multi-layer semantic information extraction: A layered encoder extracts multiple layers of semantic features from the bottom layer to the top layer of the image, enabling progressive compression of image data while preserving key information. The more abstract and generalized the high-level features, the greater the compression effect and the lower the resolution, significantly reducing the amount of transmitted data.

[0064] 2. Quantization and discretization: The quantization module maps continuous feature vectors to discrete codewords, further reducing the complexity of the data. This enables the model to learn more compact representations, improves transmission efficiency, and is suitable for unsupervised learning, compression, and generation tasks.

[0065] 3. Flexible codeword selection: Based on the channel bandwidth and noise conditions, different codeword layers can be selected for transmission. The smaller the channel bandwidth, the higher-level codewords are transmitted; the larger the channel bandwidth, the lower-level codewords are transmitted. This flexible strategy enables the system to adapt to various channel conditions and improves transmission robustness.

[0066] 4. Channel noise adaptability: Taking the additive white Gaussian noise (AWGN) channel into consideration, the quantization and decoding process are designed to effectively resist noise interference and ensure the quality of image transmission.

[0067] Based on the above embodiment, determining the target feature based on the codeword features corresponding to the multiple quantization modules includes: Step 210: Concatenate the i-th codeword feature corresponding to the i-th quantization module and the i-1-th codeword feature corresponding to the i-1-th quantization module to obtain a first codeword feature; Step 220: Concatenate the first codeword feature with the (i-2)th codeword feature corresponding to the (i-2)th quantization module to obtain a second codeword feature; Step 230: traverse from the codeword feature corresponding to the Nth quantization module to the codeword feature corresponding to the first quantization module until the target feature is obtained.

[0068] Specifically, the i-th codeword feature corresponding to the i-th quantization module is spliced with the i-1-th codeword feature corresponding to the i-1-th quantization module to obtain the first codeword feature, and then the first codeword feature is spliced with the i-2-th codeword feature corresponding to the i-2-th quantization module to obtain the second codeword feature. Finally, traverse from the codeword feature corresponding to the N-th quantization module to the codeword feature corresponding to the 1st quantization module until the target feature is obtained.

[0069] Figure 3 Schematic diagram of the receiving end in the image transmission method provided by the present invention, such as Figure 3As shown, quantized codewords code1-codeN are transmitted via noise channels 1-N, respectively. This embodiment of the present invention integrates all codeword information and inputs it into a decoder to obtain the final decoded restored image (target image). Since each codeword code1-N has a different shape, to combine them for input into the decoder, they must be transformed into a concatenated shape. Based on the concept of a pseudo-decoder, this embodiment of the present invention amplifies a high-level codeword and concatenates it with a codeword from a lower level. This concatenation result is then amplified again and concatenated with a codeword from a lower level, and so on until it is concatenated with the codeword from the lowest level. The combined concatenation of codewords from all levels is then input into the decoder. To achieve this, the present invention designs an upsampling module that directly implements upsampling through a deconvolutional network. The deconvolutional network increases the resolution of the high-level code by a factor of two, bringing it to the same size as the low-level codeword. Specifically, y = ConvTranspose2d (x), where x is the input shape c × h × w, and the output y has a shape c × 2h × 2w. The high-level codeword after the upsampling module has the same shape as the lower-level codeword. It is concatenated into a variable according to the first dimension channel and continues to be input into the upsampling module.

[0070] Since each upsampling module performs the same operation as the deconvolution network to achieve twice the feature amplification, only one upsampling module is actually needed and it can be called repeatedly.

[0071] After concatenating code1 transmitted through the last noisy channel 1 with the output of the upsampling module, the resulting vector is a quantized codeword that integrates N layers of image features, known as the target feature. This codeword has a shape of c × H / 4 × W / 4. The target feature is input to the decoder to produce the final restored image (target image). The decoder Dec consists of a concatenated convolutional network, a residual layer, an activation function, and two deconvolutional layers. Except for the last activation function, which uses Tanh, all other activation functions are Reinforced Luminance (ReLU). The convolutional network only organizes and fine-tunes the input information without changing the shape of the input data. The residual layer, defined similarly to the encoder, also does not change the shape of the input data, but further refines the features. The deconvolutional layer consists of a deconvolutional network and an activation function. The codeword resolution doubles with each upsampling. The two deconvolutional layers increase the resolution by a factor of four, gradually increasing the resolution of the feature codewords to match the original input data. Mathematically, this is expressed as y = Tanh(ConvTranspose2d(ReLU(ConvTranspose2d(ReLU(ResBlock(Conv(x))))))), where x is the input of c×H / 4×W / 4 and y is the output of C×W×H, which is the same size as the image to be transmitted.

[0072] As you can understand, the receiver uses the upsampling module and decoder to gradually amplify and concatenate the quantized codewords from different layers. Ultimately, information from all layers is combined for decoding, resulting in a high-quality restored image. This multi-layer information fusion approach effectively restores image details and semantic information. Experimental results show that under the same channel conditions, more layers improve transmission quality and result in a higher PSNR (Peak Signal-to-Noise Ratio) score for the restored image.

[0073] Based on the above embodiment, the training step of the image transmission model includes: Step 310: obtaining an initial image transmission model and a sample transmission image; Step 320: input the sample transmission image into the initial image transmission model to obtain a predicted target image output by the initial image transmission model; Step 330 : determining a target loss based on the difference between the predicted target image and the sample transmission image, and performing parameter iteration on the initial image transmission model based on the target loss to obtain the image transmission model.

[0074] Specifically, in order to better obtain the image transmission model, training can be performed based on the following steps: First, an initial image transmission model and a sample transmission image are obtained, wherein the parameters of the initial image transmission model may be preset or randomly generated, which is not specifically limited in the embodiment of the present invention.

[0075] Then, the sample transmission image is input into the initial image transmission model to obtain the predicted target image output by the initial image transmission model.

[0076] Finally, based on the difference between the predicted target image and the sample transmission image, the target loss is determined. The parameters of the initial image transmission model are iterated based on the target loss. The initial image transmission model after the parameter iteration is used as the image transmission model. The formula of the target loss is as follows:

[0077] in, represents the target loss, represents the sample transmission image, represents the predicted target image, represents the difference between the sample transmission image and the predicted target image, represents the elements of the matrix in the sample transmission image, Represents the elements of the matrix in the predicted target image, Represents the matrix size, that is and The size of the matrix formed.

[0078] It should be noted that the greater the difference between the predicted target image and the sample transmission image, the greater the target loss; the smaller the difference between the predicted target image and the sample transmission image, the smaller the target loss. The target loss can be a mean squared error loss function (MSE), which is not specifically limited in this embodiment of the present invention.

[0079] Based on the above embodiment, determining the target loss based on the difference between the predicted target image and the sample transmission image in step 330 includes: Step 331: Obtain an initial quantization module, sample coding features, and an initial dictionary set of the sample coding features; Step 332: Based on the initial quantization module, feature mapping is performed on the sample coding features and the codewords in the initial dictionary set to obtain quantized codeword features output by the initial quantization module; Step 333: determining an internal loss based on a difference between the quantized codeword feature and the sample coding feature; Step 334 , determining a transmission loss based on a difference between the predicted target image and the sample transmission image; Step 335 : Determine the target loss based on the internal loss and the transmission loss.

[0080] Specifically, an initial quantization module, sample coding features, and an initial dictionary set of sample coding features may be obtained.

[0081] After obtaining the initial quantization module, feature mapping can be performed on the sample coding features and the codewords in the initial dictionary set based on the initial quantization module to obtain the quantized codeword features output by the initial quantization module. For example, a target difference between the sample coding features and the codewords in the initial dictionary set can be determined based on the initial quantization module, a target codeword can be determined based on the target difference, and the quantized codeword features output by the initial quantization module can be determined based on the target codeword.

[0082] Then, the internal loss can be determined based on the difference between the quantized codeword features and the sample coding features. The formula for the internal loss is as follows:

[0083]

[0084] in, represents internal loss, represents the quantization error, represents the quantized codeword features, Represents the sample encoding features.

[0085] It is understandable that the quantization error generated by the transmitter during each quantization Accumulate and you get Recorded as internal loss.

[0086] Furthermore, the transmission loss (external loss) can be determined based on the difference between the predicted target image and the sample transmission image. The formula for the transmission loss is as follows:

[0087] in, Indicates transmission loss, represents the sample transmission image, represents the predicted target image, represents the difference between the sample transmission image and the predicted target image, represents the elements of the matrix in the sample transmission image, Represents the elements of the matrix in the predicted target image, Represents the matrix size, that is and The size of the matrix formed.

[0088] Finally, based on the internal loss and transmission loss, the target loss is determined. The formula for the target loss is as follows:

[0089] in, represents the target loss, Indicates transmission loss, represents internal loss, represents the weight, The value range is 0-1, and the default is 0.75.

[0090] It should be noted that the embodiment of the present invention adopts a gradient descent method to simultaneously update the parameters of the sending end and the receiving end during the training process so as to minimize the target loss function.

[0091] It should be noted that after the initial quantization module training is completed, the target dictionary set can be obtained. In the application process of the image transmission model, the trained target dictionary set can be directly applied to perform feature mapping between the coding features and the codewords in the target dictionary set.

[0092] Based on the above embodiment, step 332 includes: Step 3321: Based on the initial quantization module, determine the Euclidean distance between the sample coding feature and the codewords in the initial dictionary set, determine the target codeword based on the Euclidean distance, and determine the quantization codeword feature output by the initial quantization module based on the target codeword.

[0093] Specifically, based on the initial quantization module, the Euclidean distance between the sample coding feature and the codewords in the initial dictionary set is determined, based on the Euclidean distance, the target codeword is determined, and based on the target codeword, the quantization codeword feature output by the initial quantization module is determined.

[0094] Specifically, the main function of the quantization module is to map the input continuous feature vector to a discrete codeword. The work of the quantization module can be divided into the following steps: first, the distance between the sample coding feature and the codewords in the initial dictionary set is calculated. The vector of the sample coding feature is flattened into a two-dimensional tensor, and then the Euclidean distance is calculated with the codewords in the initial codeword set. For example, for the sample coding feature x∈R d and the codewords e∈R in the initial dictionary set d×K (where K is the number of codewords), the distance is calculated as: , here we use the vector broadcast mechanism to make the calculation more efficient. Then, according to the calculated distance, we select the codeword index that is closest to the target codeword. ,in is the kth codeword in the initial dictionary set. Then, during the training phase, the initial dictionary set is dynamically updated based on the input data. The update formula is ,in is assigned to the codeword The input sample set, is the learning rate, Is a small positive number used to avoid division by zero. Finally, the quantized output is the selected codeword. , in order to make the quantization process differentiable, the quantization output is calculated as follows: , the "detach" operation is used here, so that only the input Derivative, not the initial dictionary set Find the derivative.

[0095] It should be noted that the quantization module uses the "detach" operation to make the quantization process differentiable, which facilitates backpropagation training and improves the training efficiency and convergence speed of the image transmission model.

[0096] Based on any of the above embodiments, Figure 4 Schematic diagram of the transmitting end of the two-layer semantic information extraction, quantization and transmission method provided by the present invention. Figure 5Schematic diagram of the receiving end of the two-layer semantic information extraction quantization transmission method provided by the present invention, such as Figure 4 、 Figure 5 As shown, the image to be transmitted is input into the encoding module Enc1 to obtain the bottom encoding feature char1, cha1 is input into the encoding module Enc2 to obtain the top encoding feature char2, the top encoding feature char2 is quantized to obtain the codeword feature code2, and the codeword feature code2 is input into the pseudo decoding module Dec'2 to obtain the intermediate vector.

[0097] Furthermore, the intermediate vector is concatenated with char1 to obtain the codeword feature code1. It should be noted that the codeword feature code1 and the codeword feature code2 correspond to the noise channel 1 and the noise channel 2, respectively.

[0098] The receiving end upsamples the codeword feature code2 transmitted by the noise channel 2, and concatenates it with the codeword feature code1 transmitted by the noise channel 1 to obtain the target feature, and decodes the target feature using the decoder Dec to obtain the target image.

[0099] Based on any of the above embodiments, the embodiments of the present invention address the transmission issues of image data, taking into account challenges such as large data volume, transmission bandwidth limitations, low transmission efficiency and channel noise, and propose a concept of image layered semantics to explore deep rules, adopt different encoding methods for semantics at different levels, improve data noise resistance, and adapt to different environmental channels.

[0100] This algorithm uses deep neural networks, combined with a codec and quantization module, to process images. The codec primarily uses convolutional networks to extract semantic information while also providing compression. The quantization module compares the input vector with the embedding vector and finds the closest embedding vector as the quantized output. This invention provides a multi-layer semantic information extraction and quantization transmission scheme for images. The corresponding image transmission model consists of an encoder, decoder, quantization module, upsampling module, and pseudo-decoder, which includes convolutional layers, deconvolutional layers, and residual layers.

[0101] The image to be transmitted is first encoded at the transmitter through several encoders, extracting features layer by layer. The top-level features are quantized into codewords and transmitted over a suitable channel. A pseudo-decoder then decodes the top-level quantized codewords. The resulting intermediate vector is concatenated with the features from the next lower level and quantized into codewords, which are then transmitted over a suitable channel. This process continues until all features are concatenated with the intermediate vectors obtained from decoding the quantized codewords from the previous level and quantized into codewords. A suitable channel is then selected for transmission to the receiver. The transmitter determines the upsampling factor for the received codewords based on the layer level and uses deconvolution to upsample them to the size of the bottom-level codewords. The bottom-level codeword and all codewords are then upsampled and concatenated sequentially into intermediate vectors. The intermediate vectors are decoded to produce the decoded image.

[0102] During the training process, each quantization process will obtain a quantization error. The sum of all quantization errors is recorded as internal loss. The mean square error between the decoded image at the receiver and the original image to be transmitted at the sender is recorded as external loss. The weighted sum of the internal loss and the external loss is regarded as the target loss function, and feedback is used to guide training.

[0103] Compared to existing technologies, this method utilizes convolutional networks and codeword quantization to learn and encode data, converting it into codewords suitable for transmission across different channels. Decoding is then performed through a deconvolutional network and upsampling, achieving efficient data compression and noise-resistant transmission. This not only reduces the amount of data required for transmission but also ensures maximum preservation of image quality and detail. This method intelligently schedules the encoding and decoding process based on the channel signal-to-noise ratio (SNR), enabling more efficient utilization of network resources. This method significantly reduces pixel-level errors and significantly improves structural features and visual similarity. It is also suitable for allocating transmission data across different channels on demand when channel bandwidth is limited. For large images, this method can improve the structural features of the transmitted image (such as texture clarity, color vividness, and edge smoothness), resulting in higher visual quality, especially in channels with low SNR.

[0104] In summary, the present invention is applicable to a wide range of image data transmission scenarios, including remote sensing satellites, medical imaging, and security monitoring. In these scenarios, image data often needs to be transmitted efficiently within limited bandwidth while ensuring image quality and semantic integrity.

[0105] The design of the quantization module enables the model to be used for unsupervised learning tasks, further expanding its application scenarios.

[0106] The encoding process also achieves protocol-based encryption of data to a certain extent, enhancing the privacy and security of the data, making it more difficult to interpret and steal during transmission, reducing the loss of data directly exposed to attacks in an unsafe environment, and thus protecting the confidentiality of the data.

[0107] By discretizing the features through the quantization module, the complexity of the image transmission model is reduced, the generalization ability of the image transmission model is improved, and the consumption of computing resources is reduced.

[0108] The weighted loss function is combined with internal loss and external loss, and training is performed through gradient descent, which can quickly optimize model parameters and improve training efficiency.

[0109] The image transmission device provided by the present invention is described below. The image transmission device described below and the image transmission method described above can be referenced to each other.

[0110] Based on any of the above embodiments, the present invention provides an image transmission device, Figure 6 Schematic diagram of the structure of the image transmission device provided by the present invention, such as Figure 6 As shown, the device includes: An acquisition unit 610 is configured to acquire an image to be transmitted; A transmission unit 620 is configured to input the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0111] In the apparatus provided by the embodiments of the present invention, the quantization portion generates target features based on the coding features and intermediate features. Based on this, the quantization strategy is flexibly adjusted according to the noise characteristics of each channel. This ensures optimal subsequent decoding while meeting channel capacity requirements, thereby fully utilizing the transmission potential of each channel and improving overall transmission efficiency and quality. Furthermore, because the quantization strategy can be optimized for different noise channels, it eliminates the need to segment or transmit the image multiple times, thereby improving the overall quality of image transmission.

[0112] Based on any of the above embodiments, the quantization part includes multiple quantization modules, and the quantization strategies of the multiple quantization modules correspond to different noise channels; For any one of the plurality of serially connected encoding modules, a corresponding quantization module is configured; The quantization modules are numbered from 1 to N, and the pseudo-decoding modules are numbered from 2 to N, wherein, for any value of i from N-1 to 1, the i-th quantization module corresponds to the i+1-th pseudo-decoding module; When i is equal to N, the i-th quantization module is used to quantize and encode the coding features of the i-th coding module to obtain the top-level codeword features; the i-th pseudo-decoding module is used to decode the top-level codeword features corresponding to the i-th quantization module to obtain the top-level intermediate features; For the coding feature output by the i-th encoder, when i is not equal to N, the i-th encoding module performs image encoding on the image to be transmitted to obtain the i-th coding feature, and the i-th quantization module splices the intermediate feature of the i+1-th pseudo decoding module with the i-th coding feature to obtain the i-th codeword feature; The iterative unit is configured to iteratively execute steps from i=N-1 to i=1 to obtain codeword features corresponding to multiple quantization modules, and determine the target feature based on the codeword features corresponding to the multiple quantization modules.

[0113] Based on any of the above embodiments, the iteration unit is specifically configured to: Concatenate the i-th codeword feature corresponding to the i-th quantization module and the i-1-th codeword feature corresponding to the i-1-th quantization module to obtain the first codeword feature; Concatenate the first codeword feature with the (i-2)th codeword feature corresponding to the (i-2)th quantization module to obtain a second codeword feature; Traverse from the code word feature corresponding to the Nth quantization module to the code word feature corresponding to the 1st quantization module until the target feature is obtained.

[0114] Based on any of the above embodiments, a model training unit is further included, and the model training unit specifically includes: An acquisition model and sample unit is used to acquire an initial image transmission model and a sample transmission image; An input unit, configured to input the sample transmission image into the initial image transmission model to obtain a predicted target image output by the initial image transmission model; A parameter iteration unit is used to determine a target loss based on a difference between the predicted target image and the sample transmission image, and perform parameter iteration on the initial image transmission model based on the target loss to obtain the image transmission model.

[0115] Based on any of the above embodiments, the parameter iteration unit specifically includes: An acquisition module unit, configured to acquire an initial quantization module, a sample coding feature, and an initial dictionary set of the sample coding feature; a feature mapping unit, configured to perform feature mapping on the sample coding features and the codewords in the initial dictionary set based on the initial quantization module, to obtain quantized codeword features output by the initial quantization module; an internal loss determining unit, configured to determine an internal loss based on a difference between the quantized codeword feature and the sample coding feature; a transmission loss determining unit, configured to determine a transmission loss based on a difference between the predicted target image and the sample transmission image; The target loss determining unit is configured to determine the target loss based on the internal loss and the transmission loss.

[0116] Based on any of the above embodiments, the feature mapping unit is specifically configured to: Based on the initial quantization module, a Euclidean distance between the sample coding feature and the codewords in the initial dictionary set is determined, a target codeword is determined based on the Euclidean distance, and a quantized codeword feature output by the initial quantization module is determined based on the target codeword.

[0117] Based on any of the above embodiments, the encoder includes a convolutional layer, a residual layer and an activation layer in sequence; The pseudo decoder includes a convolution layer, a residual layer, a deconvolution layer and an activation layer in sequence.

[0118] Based on any of the above embodiments, the decoder includes a convolution layer, a residual layer, an activation layer, a deconvolution layer and a deconvolution layer in sequence.

[0119] Figure 7 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call logic instructions in the memory 730 to execute an image transmission method, which includes: obtaining an image to be transmitted; inputting the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; the image transmission model includes an encoder, a quantization part, a pseudo decoder, and a decoder; the encoder includes multiple encoding modules connected in series, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; the encoder is configured to encode the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo decoder is configured to decode the codeword feature corresponding to each quantization module based on each pseudo decoding module to obtain each intermediate feature; the quantization part is configured to obtain a target feature based on the encoding features and the intermediate features; and the decoder is configured to decode the target feature to obtain the target image.

[0120] Furthermore, the logic instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0121] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image transmission method provided by the above-mentioned methods, which includes: obtaining an image to be transmitted; inputting the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; the image transmission model includes an encoder, a quantization part, a pseudo-decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo-decoder includes multiple pseudo-decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; the encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0122] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image transmission method provided by the above-mentioned methods, the method comprising: obtaining an image to be transmitted; inputting the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; the image transmission model comprises an encoder, a quantization part, a pseudo-decoder and a decoder; the encoder comprises a plurality of encoding modules connected in series, and the pseudo-decoder comprises a plurality of pseudo-decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; the encoder is used to perform image encoding on the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo-decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo-decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the each encoding feature and the each intermediate feature; the decoder is used to decode the target feature to obtain the target image.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0124] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image transmission method, characterized in that: include: Obtaining the image to be transmitted; Inputting the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to encode the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the encoding features and the intermediate features; The decoder is used to decode the target feature to obtain the target image.

2. The image transmission method according to claim 1, wherein: The quantization part includes a plurality of quantization modules, and the quantization strategies of the plurality of quantization modules correspond to different noise channels; For any one of the plurality of serially connected encoding modules, a corresponding quantization module is configured; The quantization modules are numbered from 1 to N, and the pseudo-decoding modules are numbered from 2 to N, wherein, for any value of i from N-1 to 1, the i-th quantization module corresponds to the i+1-th pseudo-decoding module; When i is equal to N, the i-th quantization module is used to quantize and encode the coding features of the i-th coding module to obtain the top-level codeword features; the i-th pseudo-decoding module is used to decode the top-level codeword features corresponding to the i-th quantization module to obtain the top-level intermediate features; For the coding feature output by the i-th encoder, when i is not equal to N, the i-th encoding module performs image encoding on the image to be transmitted to obtain the i-th coding feature, and the i-th quantization module splices the intermediate feature of the i+1-th pseudo decoding module with the i-th coding feature to obtain the i-th codeword feature; Iteratively perform steps from i=N-1 to i=1 to obtain codeword features corresponding to multiple quantization modules, and determine the target feature based on the codeword features corresponding to the multiple quantization modules.

3. The image transmission method according to claim 2, wherein: The determining the target feature based on the codeword features corresponding to the multiple quantization modules includes: Concatenate the i-th codeword feature corresponding to the i-th quantization module and the i-1-th codeword feature corresponding to the i-1-th quantization module to obtain the first codeword feature; Concatenate the first codeword feature with the (i-2)th codeword feature corresponding to the (i-2)th quantization module to obtain a second codeword feature; Traverse from the code word feature corresponding to the Nth quantization module to the code word feature corresponding to the 1st quantization module until the target feature is obtained.

4. The image transmission method according to claim 2, wherein: The training steps of the image transmission model include: Obtaining an initial image transmission model and a sample transmission image; Inputting the sample transmission image into the initial image transmission model to obtain a predicted target image output by the initial image transmission model; Based on the difference between the predicted target image and the sample transmission image, a target loss is determined, and parameters of the initial image transmission model are iterated based on the target loss to obtain the image transmission model.

5. The image transmission method according to claim 4, characterized in that: The determining of the target loss based on the difference between the predicted target image and the sample transmission image includes: Obtaining an initial quantization module, sample coding features, and an initial dictionary set of the sample coding features; Based on the initial quantization module, feature mapping is performed on the sample coding feature and the codewords in the initial dictionary set to obtain the quantized codeword feature output by the initial quantization module; determining an internal loss based on a difference between the quantized codeword feature and the sample coding feature; determining a transmission loss based on a difference between the predicted target image and the sample transmission image; The target loss is determined based on the internal loss and the transmission loss.

6. The image transmission method according to claim 5, characterized in that: The performing feature mapping on the sample coding feature and the codewords in the initial dictionary set based on the initial quantization module to obtain the quantized codeword features output by the initial quantization module includes: Based on the initial quantization module, a Euclidean distance between the sample coding feature and the codewords in the initial dictionary set is determined, a target codeword is determined based on the Euclidean distance, and a quantized codeword feature output by the initial quantization module is determined based on the target codeword.

7. The image transmission method according to any one of claims 1 to 6, characterized in that: The encoder includes a convolutional layer, a residual layer and an activation layer in sequence; The pseudo decoder includes a convolution layer, a residual layer, a deconvolution layer and an activation layer in sequence.

8. The image transmission method according to any one of claims 1 to 6, characterized in that: The decoder includes a convolutional layer, a residual layer, an activation layer, a deconvolution layer and a deconvolution layer in sequence.

9. An image transmission device, characterized in that: include: An acquisition unit, configured to acquire an image to be transmitted; A transmission unit, configured to input the image to be transmitted into an image transmission model to obtain a target image output by the image transmission model; The image transmission model includes an encoder, a quantization part, a pseudo decoder and a decoder; the encoder includes multiple serially connected encoding modules, and the pseudo decoder includes multiple pseudo decoding modules; the quantization strategy of the quantization part corresponds to different noise channels; The encoder is used to encode the image to be transmitted based on each encoding module to obtain each encoding feature; the pseudo decoder is used to decode the codeword features corresponding to each quantization module based on each pseudo decoding module to obtain each intermediate feature; the quantization part is used to obtain the target feature based on the encoding features and the intermediate features; The decoder is used to decode the target feature to obtain the target image.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the image transmission method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image transmission method according to any one of claims 1 to 8 is implemented.

Images