End-to-end joint optimization method for image compression transmission application

Abstract

According to the end-to-end joint optimization method for the image compression transmission application, deep learning is combined with a traditional image coding technology, the neural network is optimized step by step by using different training strategies, the problem that the neural network gradient is limited by non-conductance and is difficult to carry out back propagation is solved, and the method is suitable for large-scale popularization and application. The current situation that a similar compatible image codec only recovers a compressed image from the perspective of post-processing is improved, an image compression process is no longer regarded as an image degradation problem through a fixed image prior model, the compression coding efficiency in an image transmission process is improved, spatial domain redundancy in the image is removed to a greater extent, and the image quality is improved. The problem that the image quality is reduced in a low-bandwidth environment is effectively solved, and the method can be applied to image compression transmission.

Description

Technical Field

[0001] The present invention belongs to the technical field of image transmission, and in particular relates to an end-to-end joint optimization method for image compression transmission applications. Background Art

[0002] The 21st century is an era of digitization, networking and informatization. People are exchanging information through various high-tech products, and the requirements for the speed and security of information transmission are getting higher and higher. Information exchange has gradually changed from pure text communication such as WeChat and QQ to video image communication such as Douyin and Kuaishou. The change in communication methods has made image processing particularly important. Image compression refers to the process of reducing the amount of data required for a given location. The image is compressed to remove redundancy in the encoding, spatial and temporal redundancy, and irrelevant information in the image, thereby increasing the transmission rate and saving storage space. Therefore, traditional image coding standards such as JPEG, BPG, HEVC, and AVS are widely used.

[0003] In actual task applications, we often use acceleration algorithms deployed in hardware to obtain higher flexibility and processing speed than software. However, in the field of image compression algorithms, there are two contradictions between software and hardware: software algorithms with better performance often have greater computational complexity, resulting in hardware with relatively small computing resources being unable to meet algorithm requirements; at the same time, the faster iterative optimization of software-side algorithms puts a lot of pressure on hardware devices that are already difficult to update. Since the operators used in neural networks are gradually adapted to hardware at this stage, this invention studies the improvement of overall image compression performance through a lightweight end-to-end framework based on deep learning without changing the core image codec.

[0004] In summary, in order to solve the contradiction between software and hardware in the current algorithm application, it is necessary to seek an end-to-end joint optimization method for image compression and transmission applications to integrate deep learning and traditional image coding technology into an overall framework. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an end-to-end joint optimization method for image compression transmission applications. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] An end-to-end joint optimization method for image compression transmission applications includes:

[0007] S100, constructing a feature extraction network at the front end and a feature reconstruction network at the back end of a codec in the image low-bandwidth transmission system, and pre-training the feature extraction network and the feature reconstruction network separately to obtain a pre-trained feature extraction network and a pre-trained feature reconstruction network;

[0008] S200, designing a codec neural network simulating the codec, and setting it between the pre-trained feature extraction network and the feature reconstruction network, and then training the feature extraction network and the feature reconstruction network by fixing the parameters of the trained codec neural network to obtain an initially trained feature extraction network and feature reconstruction network;

[0009] S300, fixing the parameters in the initially trained feature extraction network to make it a filter, and inputting the original image into the filter, so as to perform joint optimization training on the initially trained feature reconstruction network using the original image, and obtain a jointly optimized feature reconstruction network;

[0010] S400, fixing the parameters of the jointly optimized feature reconstruction network, and inputting the original image into the feature extraction network to perform joint optimization training on it until the parameters converge, thereby obtaining a final image low-bandwidth transmission system.

[0011] Beneficial effects:

[0012] The present invention provides an end-to-end joint optimization method for image compression and transmission applications, which combines deep learning with traditional image coding technology, and gradually optimizes the neural network using different training strategies, thereby solving the problem that the gradient of the neural network is difficult to backpropagate due to its non-differentiability, improving the current situation that similar compatible image codecs only restore compressed images from a post-processing perspective, no longer regarding the image compression process as an image degradation problem through a fixed image prior model, improving the compression coding efficiency during image transmission, removing the internal spatial redundancy of the image to a greater extent, and effectively solving the problem of image quality degradation in a low-bandwidth environment, and can be applied to image compression transmission.

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of an end-to-end joint optimization method for image compression transmission application provided by the present invention;

[0015] Figure 2 is the original image;

[0016] Figure 3 is an image compressed using an image codec;

[0017] Figure 4 are images compressed using a method that has not been jointly trained;

[0018] Figure 5 An image compressed and transmitted using the method of the present invention;

[0019] Figure 6It is a schematic diagram of rate-distortion curves of comparative tests between the present invention and existing image coding standards. DETAILED DESCRIPTION

[0020] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0021] The image compression and transmission application system of the present invention comprises the following steps: at the output end, the input image is passed through a sampling feature extraction network at the front end to obtain a dimension reduction feature map having visual information, and the feature map is input into an image encoder to obtain a target image encoding file; the encoding file is sent to a receiving end in the form of a data stream, and the receiving end obtains a decoded feature map through an image decoder; the decoded feature map is passed through a feature reconstruction network at the back end to obtain a final dimension increase reconstructed image.

[0022] like Figure 1 As shown, the present invention provides an end-to-end joint optimization method for image compression transmission application, including:

[0023] S100, constructing a feature extraction network at the front end and a feature reconstruction network at the back end of a codec in the image low-bandwidth transmission system, and pre-training the feature extraction network and the feature reconstruction network separately to obtain a pre-trained feature extraction network and a pre-trained feature reconstruction network;

[0024] This step builds the front-end feature extraction network Net enc And the back-end feature reconstruction network Net dec , through pre-training, the image low-bandwidth transmission system initially has the image transmission function that is compatible with the image codec.

[0025] In a specific embodiment of the present invention, S100 includes:

[0026] S110, constructing a feature extraction network at the front end of a codec in the image low-bandwidth transmission system and a feature reconstruction network at the back end;

[0027] S120, initializing parameters of the feature extraction network and initializing parameters of the feature reconstruction network;

[0028] S130, for the feature extraction network, the original image Image inp The input feature extraction network is encoded and decoded by the codec, and then upsampled by bicubic interpolation to obtain the first output image Image enc ;Use the original image Image inp With the first output image Image enc, calculating a first loss function; performing back propagation using the first loss function to update the gradient of the feature extraction network until the parameters converge, thereby obtaining a pre-trained feature extraction network;

[0029] Among them, the first loss function is expressed by formula:

[0030]

[0031] Image enc =Net enc (Image inp );

[0032] In the formula, Coder() represents the codec, Up bicubic () is bicubic interpolation upsampling, Net enc is the feature extraction network; i is the pixel number in the image, and m is the total number of pixels in the image;

[0033] S140, for the feature reconstruction network, the original image Image inp After downsampling and codec, it is input into the feature reconstruction network to obtain the output image Image down , using the original image Image inp And the output image Image down , calculate the second loss function; perform back propagation according to the second loss function to update the gradient of the feature reconstruction network until the parameters converge, and obtain a pre-trained feature reconstruction network.

[0034] The second loss function is expressed as follows:

[0035]

[0036] Image down =Down bicubic (Coder(Image inp ));

[0037] In the formula, Net dec Represents feature reconstruction network, Down bicubic Indicates downsampling.

[0038] This implementation method completes the pre-training of the networks at both ends, so that the framework of the low-bandwidth image transmission system based on deep learning is completed, and it initially has the image transmission function that is compatible with the image codec.

[0039] S200, designing a codec neural network simulating the codec, and setting it between the pre-trained feature extraction network and the feature reconstruction network, and then training the feature extraction network and the feature reconstruction network by fixing the parameters of the trained codec neural network to obtain an initially trained feature extraction network and feature reconstruction network;

[0040] In a specific implementation of the present invention, S200 includes:

[0041] S210, simulating the codec to design a codec neural network;

[0042] S220, training the codec neural network to obtain a trained codec neural network, and setting the trained codec neural network between the pre-trained feature extraction network and the feature reconstruction network;

[0043] S230, fixing the parameters of the trained encoding and decoding neural network, and using the original image Image inp The pre-trained feature extraction network and feature reconstruction network are trained to obtain the initially trained feature extraction network and feature reconstruction network.

[0044] This embodiment designs a codec neural network Net that simulates the codec cod , the codec neural network simulates the effect of codec on image data. The intra-frame prediction inside the image codec is mainly composed of 16×16 macroblocks and their internal sub-blocks. cod The image regional blur produced by the encoding and decoding process is simulated by multi-scale residual blocks, and the simulated encoding and decoding network Net cod The replacement codec is trained so that the gradient can be differentiable with respect to the parameters of the downsampling network during training, which improves the adaptability of the entire framework to the image codec and speeds up the iteration rate.

[0045] The image transmission process is expressed by the following formula:

[0046] F((θ1,θ2),x)=Net dec (θ2,Coder(Net enc (θ1,Image inp )),Image inp );

[0047] Among them, θ1 and θ2 are the feature reconstruction network Net dec and feature extraction network Net enc The parameters of Coder() represent compatible image codecs, Image inp is the input image. Input image Image inpAfter feature extraction network Net enc , image codec Coder() and feature reconstruction network Net dec The final result is a reconstructed image, which is an end-to-end image compression framework.

[0048] Since the image codec contains a non-differentiable quantization process, back propagation cannot be performed simultaneously in both ends of the network. In order to solve this problem, the present invention designs a joint optimization algorithm, which divides the optimization difference of the network framework structure into two sub-problems, that is, fix any one of (θ1, θ2) as a constant to train the other one, and the formula is as follows:

[0049]

[0050] S300, fixing the parameters in the initially trained feature extraction network to make it a filter, and inputting the original image into the filter, so as to perform joint optimization training on the initially trained feature reconstruction network using the original image, and obtain a jointly optimized feature reconstruction network;

[0051] In a specific implementation of the present invention, S300 includes:

[0052] S310, fixing the parameters in the initially trained feature extraction network to make it a filter;

[0053] S320, the original image Image inp Input into the filter, and then pass through the codec and the feature reconstruction network to obtain the third output image Image mid ;

[0054] S330, using the original image Image inp And the third output image Image mid , calculate the third loss function;

[0055] Design feature reconstruction network Net dec The main body of the loss function includes perceptual loss and mean square error loss. Using the feature extraction network Net enc Pre-training final image Image enc With input Image inp Train feature reconstruction network Net dec , through the perceptual loss PerceptualLoss and the mean square error loss MSELoss combined to back propagate and update the Net dec Gradient, perceptual loss P(x,y) and mean square error loss MSE(x) formulas are as follows

[0056]

[0057] L u =MSE(Image mid ,Image imp )+σ1×P(Image mid ,Image imp )

[0058] Where σ1=0.001 is the perceptual loss coefficient, where F i (x) Use the Vgg network to calculate the position of the input image and the target image in the feature space, j is the feature representation of the jth layer in the Vgg network, and N is the total number of feature layers in the Vgg network. Image mid The final generated image for this stage of training;

[0059] Design a gradient-constrained loss function in the feature reconstruction network. In the process of updating the θ1 parameter, due to the feature reconstruction network Net dec It is the main part of the end-to-end image compression framework, which includes most of the model's computational complexity and parameter size. dec The output image of is the final image, so the loss function based on gradient constraint is designed to dec Make more refined adjustments.

[0060] Affected by the intra-frame prediction inside the image codec, in order to solve the serious block effect and regional blurring of the image through the codec, the feature reconstruction network Net dec Affects the final image distortion. dec The training introduces gradient constraints, relying on the edge gradient comparison of the output image and the input image to improve image clarity, repair sharp edges, and eliminate the influence of the image codec during image transmission. The loss function of the gradient is shown in the following formula:

[0061]

[0062] Gdl=σ2×MAE(OUT grad ,GT grad )

[0063] Among them, σ2=0.1 is the gradient loss coefficient, OUT grad ,GT grad is the gradient map of the output image and the original image, calculated according to the following formula:

[0064]

[0065] in, Both are 3×3 unit matrices, used to capture the horizontal and vertical gradient distribution of the image. Represents a matrix multiplication operation.

[0066] S340, using the third loss function, back-propagating and updating the gradient of the initially trained feature reconstruction network until the parameters converge, to obtain a jointly optimized feature reconstruction network.

[0067] The post-feature reconstruction network is trained, and through the design of the above loss function, the feature reconstruction network Net dec The optimized third loss function is shown in the following formula:

[0068]

[0069] L u =MSE(Image mid ,Image inp )+σ1×P(Image mid ,Image inp );

[0070]

[0071] Gdl=σ2×MAE(OUT grad ,GT grad );

[0072]

[0073] In the formula, θ1, θ2 are the feature reconstruction network Net dec and feature extraction network Net enc Parameters, F(θ1,x) means that the θ2 parameter is regarded as a constant, σ1 is the perceptual loss function; i is the pixel number in the image, m is the total number of pixels in the image, σ2 is the gradient loss coefficient, OUT grsf Output image Image mid Gradient map, GT grad is the gradient map of the original image, F j (x) is the position of the input image x in the feature space calculated using the Vgg network, j is the feature representation of the jth layer in the Vgg network, and N is the total number of feature layers in the Vgg network.

[0074] In the Net dec The Adam algorithm is used for gradient update in a single network training, where the algorithm parameters are set to α=0001, β1=0.9, β1=0.999, ∈=10 -8 In the experiment, the batch size is set to 128, the training epoch is 50, and the learning rate decays exponentially from 0.005 to 0.0005 and 0.00005 in the 15th and 30th epochs respectively.

[0075] S400, fixing the parameters of the jointly optimized feature reconstruction network, and inputting the original image into the feature extraction network to perform joint optimization training on it until the parameters converge, thereby obtaining a final image low-bandwidth transmission system.

[0076] Among them, in the joint optimization training process, the gradient is updated through multiple training rounds, and the Adam algorithm is used to update the gradient in each training round.

[0077] In a specific implementation of the present invention, S400 includes:

[0078] S310, fixing the parameters in the initially trained feature reconstruction network to make it an upsampling filter;

[0079] S320, the original image Image inp Input into the initially trained feature extraction network, and then pass through the codec and the up-sampling filter to obtain a fourth output image;

[0080] S330, using the original image Image inp and a fourth output image, calculating a fourth loss function;

[0081] Design a back-end equivalent upsampling filter with fixed parameters and a feature extraction network Net that maintains image visual information enc Training strategy. In the process of updating the θ2 parameter, since Net enc It is not the final output of the whole framework, so it is impossible to directly construct a loss function with the input image. The whole framework must be trained uniformly. The image codec has a non-differentiable property, so the feature reconstruction network Net is used. dec The feature extraction network is trained based on the input and output characteristics of

[0082] When the θ1 parameter is regarded as a constant, the reconstruction network can be equivalent to an upsampling filter with the encoder and decoder, that is, At this time, when the quality of the reconstructed image x is higher and the information density is greater, the output image effect through the equivalent upsampling filter is better. inp The constructed loss function should also be smaller, so the feature extraction network can be optimized by outputting the constructed loss function through the overall framework.

[0083] Because Net encThe overall structure is small and only supports equivalent optimization. The present invention uses MAEloss to construct the main body of the loss function. At the same time, in order to make the feature extraction network visually appear as a reduced image obtained by reducing the resolution of the image to be transmitted as much as possible, rather than a feature image lacking visual information after some spatial domain transformation, the present invention guides the feature extraction network to maintain the visual information of the image to be transmitted through an interpolation downsampling method. The fourth loss function is expressed by the formula:

[0084] L d =MAE(Up(x),y)+σ3×MAE(x,Down(y)));

[0085]

[0086] In the formula, x=Net enc (θ1,Image inp ), σ3 is the visual information maintenance coefficient, Down() is the bicubic interpolation downsampling, Indicates that the parameter θ1 is fixed, and y represents the original image Image inp .

[0087] S340, using the fourth loss function, back-propagating and updating the gradient of the initially trained feature extraction network until the parameters converge, to obtain a jointly optimized feature extraction network and feature reconstruction network;

[0088] S350, using the image low-bandwidth transmission system consisting of a jointly optimized feature extraction network, a codec, and a jointly optimized feature reconstruction network as the final image low-bandwidth transmission system.

[0089] When training the feature extraction network, in Net enc In a single network training, the Adam algorithm is used to update the gradient, and the algorithm parameters are set the same as Net dec The training round epoch is 25, and the learning rate decays from 0.001 to 0.0001 and 0.00001 in the 10th and 20th rounds respectively.

[0090] The pseudo code of the joint optimization training strategy in S300 and S400 is as follows:

[0091]

[0092] In order to more clearly illustrate the optimization performance of the present invention, the same image is compressed using image codec, a method without joint training, a traditional iterative training method and the present invention method. Figure 2 After compressing the image using the method without joint training, the output image is shown in Figure 3After the image is compressed using the method of the present invention, the output image is shown in Figure 4 .pass Figure 2 , Figure 3 , Figure 4 and Figure 5 It can be seen from the comparison that the image reconstructed by the method of the present invention has clearer details, higher edge sharpness and less residual noise than the results of the other two methods.

[0093] The end-to-end joint optimization method for image transmission neural network tasks provided by the present invention solves the problem that the neural network gradient is difficult to backpropagate due to its non-differentiability, improves the current situation that similar compatible image codecs only restore compressed images from a post-processing perspective, and no longer regards the image compression process as an image degradation problem through a fixed image prior model, providing a practical idea for solving the software and hardware contradictions in the current algorithm application.

[0094] Below, rate-distortion curves composed of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used to quantitatively compare and evaluate the performance of the end-to-end joint optimization method for image compression and transmission applications proposed in the present invention, the traditional end-to-end iterative optimization method, and the existing image coding standards.

[0095] (1) Peak signal-to-noise ratio

[0096] The mathematical expression of peak signal-to-noise ratio PSNR is:

[0097]

[0098] Among them, I(i,j) is the pixel value at the coordinate (i,j) of the original image, I0(i,j) is the pixel value at the coordinate (i,j) of the reconstructed image, M is the length of the image, and N is the width of the image.

[0099] The larger the PSNR value, the better the noise suppression.

[0100] (2) Structural similarity

[0101] The mathematical expression of structural similarity SSIM is:

[0102] SSIM(X,Y)=[I(X,Y)] α [c(X,Y)] τ ·[s(X,Y)] γ

[0103] Among them, the brightness comparison item μ X and μ Y Respectively represent the mean of all pixel values ​​of the original image X and the reconstructed image Y, C1 is a constant, α is the brightness comparison index; contrast comparison index σ X and σ Y They represent the variance of the image X without noise pollution and the image Y after correction, C2 is a constant, τ is the contrast comparison index; the structural comparison index σ XY is the covariance of the image X without noise pollution and the corrected image Y, C3 is a constant, and γ is the structural comparison term index.

[0104] The larger the SSIM value, the higher the structural similarity between the corrected image and the original scene, and the better the detail preservation ability.

[0105] The experimental results compared with those without special optimization methods are shown in Table 1.

[0106] Table 1 Comparison of results of different training methods under quantization parameter QP=27

[0107]

[0108] Table 2 Comparison of results of different training methods under quantization parameter QP=31

[0109]

[0110] The rate distortion curve of the comparison test between the present invention and the existing image coding standard is shown in Figure 6 .

[0111] From Tables 1, 2 and Figure 6 visible:

[0112] (1) The peak signal-to-noise ratio (PSNR) and structural similarity coefficient (SSIM) of the image after the method proposed in the present invention are significantly higher than those of the same network without joint training and the results of traditional end-to-end iterative training, indicating that the image quality reconstructed by the method of the present invention is better.

[0113] (2) The image reconstructed by the method proposed in the present invention has a higher peak signal-to-noise ratio (PSNR) and structural similarity coefficient (SSIM) at the same bit rate, indicating that the image reconstructed by the method proposed in the present invention has better image quality after compressed transmission at the same bandwidth, or in other words, the image of the same quality uses less bandwidth.

[0114] The above results fully demonstrate that the method of the present invention reconstructs better image quality, retains more structural information of the original scene, has better performance and structure preservation characteristics, and is more suitable for image compression and transmission applications.

[0115] In summary, the end-to-end joint optimization method for image compression and transmission applications proposed in the present invention integrates deep learning and traditional image coding technology into an overall framework, while retaining structural information such as edges and details of the original image to the greatest extent, providing a practical idea for solving the software and hardware contradictions in the current algorithm applications.

[0116] It is worth noting that the terms "first" and "second" in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0117] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps.

[0118] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. An end-to-end joint optimization method for image compression and transmission applications, characterized in that: include: S100, constructing a feature extraction network at the front end and a feature reconstruction network at the back end of a codec in the image low-bandwidth transmission system, and pre-training the feature extraction network and the feature reconstruction network separately to obtain a pre-trained feature extraction network and a pre-trained feature reconstruction network; S200, designing a codec neural network simulating the codec, and setting it between the pre-trained feature extraction network and the feature reconstruction network, and then training the feature extraction network and the feature reconstruction network by fixing the parameters of the trained codec neural network to obtain an initially trained feature extraction network and feature reconstruction network; S300, fixing the parameters in the initially trained feature extraction network to make it a filter, and inputting the original image into the filter, so as to perform joint optimization training on the initially trained feature reconstruction network using the original image, and obtain a jointly optimized feature reconstruction network; S400, fixing the parameters of the jointly optimized feature reconstruction network, and inputting the original image into the feature extraction network to perform joint optimization training on it until the parameters converge, thereby obtaining a final image low-bandwidth transmission system.

2. The end-to-end joint optimization method for image compression and transmission applications according to claim 1, characterized in that: S100 includes: S110, constructing a feature extraction network at the front end of a codec in the image low-bandwidth transmission system and a feature reconstruction network at the back end; S120, initializing parameters of the feature extraction network and initializing parameters of the feature reconstruction network; S130, for the feature extraction network, the original image Image inp The input feature extraction network is encoded and decoded by the codec, and then upsampled by bicubic interpolation to obtain the first output image Image enc ;Use the original image Image inp With the first output image Image enc , calculating a first loss function; performing back propagation using the first loss function to update the gradient of the feature extraction network until the parameters converge, thereby obtaining a pre-trained feature extraction network; S140, for the feature reconstruction network, the original image Image inp After downsampling and codec, it is input into the feature reconstruction network to obtain the output image Image down , using the original image Image inp And the output image Image down , calculate the second loss function; perform back propagation according to the second loss function to update the gradient of the feature reconstruction network until the parameters converge, and obtain a pre-trained feature reconstruction network.

3. The end-to-end joint optimization method for image compression and transmission applications according to claim 2, characterized in that: The first loss function is expressed as: Image enc =Net enc (Image inp ) In the formula, Coder() represents the codec, Up bicubic () is bicubic interpolation upsampling, Net enc It is the feature extraction network; i is the pixel number in the image, and m is the total number of pixels in the image; The second loss function is expressed as: Image down =Downbicubic(Coder(Image inp )) In the formula, Net dec Represents feature reconstruction network, Down bicubic Indicates downsampling.

4. The end-to-end joint optimization method for image compression and transmission applications according to claim 1, characterized in that: S200 includes: S210, simulating the codec to design a codec neural network; S220, training the codec neural network to obtain a trained codec neural network, and setting the trained codec neural network between the pre-trained feature extraction network and the feature reconstruction network; S230, fixing the parameters of the trained encoding and decoding neural network, and using the original image Image inp The pre-trained feature extraction network and feature reconstruction network are trained to obtain the initially trained feature extraction network and feature reconstruction network.

5. The end-to-end joint optimization method for image compression and transmission applications according to claim 1, characterized in that: S300 includes: S310, fixing the parameters in the initially trained feature extraction network to make it a filter; S320, the original image Image inp Input into the filter, and then pass through the codec and the feature reconstruction network to obtain the third output image Image mid ; S330, using the original image Image inp And the third output image Image mud , calculate the third loss function; S340, using the third loss function, back-propagating and updating the gradient of the initially trained feature reconstruction network until the parameters converge, to obtain a jointly optimized feature reconstruction network.

6. The end-to-end joint optimization method for image compression and transmission applications according to claim 5, characterized in that: The third loss function is expressed by the formula: L u =MSE(Image mid ,Image inp )+σ1×P(Image mid ,Image inp ); Gdl=σ2×MAE(OUTi grad ,GT grad ); In the formula, θ1, θ2 are the feature reconstruction network Net dec and feature extraction network Net enc Parameters, F(θ1,x) means that the θ2 parameter is regarded as a constant, σ1 is the perceptual loss function; i is the pixel number in the image, m is the total number of pixels in the image, σ2 is the gradient loss coefficient, OUT grad Output image Image mid Gradient map, GT grad is the gradient map of the original image, F j (x) is the position of the input image x in the feature space calculated using the Vgg network, j is the feature representation of the jth layer in the Vgg network, and N is the total number of feature layers in the Vgg network.

7. The end-to-end joint optimization method for image compression and transmission applications according to claim 1, characterized in that: S400 includes: S310, fixing the parameters in the initially trained feature reconstruction network to make it an upsampling filter; S320, the original image Image inp Input into the initially trained feature extraction network, and then pass through the codec and the up-sampling filter to obtain a fourth output image; S330, using the original image Image inp and a fourth output image, calculating a fourth loss function; S340, using the fourth loss function, back-propagating and updating the gradient of the initially trained feature extraction network until the parameters converge, to obtain a jointly optimized feature extraction network and feature reconstruction network; S350, using the image low-bandwidth transmission system consisting of a jointly optimized feature extraction network, a codec, and a jointly optimized feature reconstruction network as the final image low-bandwidth transmission system.

8. The end-to-end joint optimization method for image compression and transmission applications according to claim 7, characterized in that: The fourth loss function is expressed by formula: l d =MAE(Up(x),y)+σ3×MAE(x,Down(y))); In the formula, x=Net enc (θ1,Image inp ), σ3 is the visual information maintenance coefficient, Down() is the bicubic interpolation downsampling, Indicates that the parameter θ1 is fixed, and y represents the original image Image inp .

9. The end-to-end joint optimization method for image compression and transmission applications according to claim 1, characterized in that: During the joint optimization training of S300 and S400, the gradient is updated through multiple training rounds, and the Adam algorithm is used to update the gradient in each training round.

Images