Training method and device for deep learning codec network

Abstract

The present invention discloses a training method and device for a deep learning codec network. A removable convolution-based auxiliary module for auxiliary training is introduced in the training stage. The deconvolved features of the decoding part are re-convolved to extract the reconstructed intermediate features. The feature loss is calculated with the down-sampled features obtained by convolution with the final reconstructed image. The feature loss has an improvement effect on improving PSNR and MS-SSIM indicators. Since the influence of the intermediate layer features on the reconstructed image is fully considered, the effect of image reconstruction is also improved.

Description

Technical Field

[0001] The present invention relates to the field of image coding and decoding technology, and in particular to a training method and device for a deep learning coding and decoding network. Background Art

[0002] At present, the image encoding and decoding model based on deep learning in academia has surpassed the traditional JPEG, JPEG2000, BPG, H265, and H266 in terms of BPP (pixel depth), PSNR (peak signal-to-noise ratio), MS-SSIM (multi-level structural similarity) and various subjective indicators.

[0003] At present, most deep learning codec networks are trained through rate-distortion loss, but the training effect is not good, and the image reconstruction effect of deep learning codec networks after training needs to be improved. The general rate-distortion loss does not make good use of the intermediate features of the image decoding layer, resulting in unsatisfactory image decoding quality. Summary of the invention

[0004] The purpose of the present invention is to provide a training method and device for a deep learning codec network, which can improve the effect of image reconstruction.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A training method for a deep learning codec network, the deep learning codec network comprising: an encoding part, a super prior probability model and a decoding part, including:

[0007] The first stage: setting multiple convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network;

[0008] The second stage: remove the convolution module and all convolution-based auxiliary modules, input the original image into the deep learning codec network, and calculate the rate-distortion loss as the loss function of the second stage to train the deep learning codec network trained in the first stage.

[0009] A training device for a deep learning codec network, the deep learning codec network comprising: a coding part, a super prior probability model and a decoding part, including:

[0010] The first training unit is applied to the first stage, which includes: setting a plurality of convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, and setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all the convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network;

[0011] The second training unit is applied to the second stage, and the second stage includes: removing the convolution module and all convolution-based auxiliary modules, inputting the original image into the deep learning codec network, and calculating the rate-distortion loss as the loss function of the second stage to train the deep learning codec network.

[0012] A processing device, comprising: one or more processors; a memory for storing one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0014] A readable storage medium stores a computer program, which implements the above method when the computer program is executed by a processor.

[0015] It can be seen from the technical solution provided by the present invention that a removable convolution-based auxiliary module is introduced in the training stage, and the deconvolved features of the decoding part are re-convolved to extract the reconstructed intermediate features, and the feature loss is calculated with the down-sampled features obtained by convolution with the final reconstructed image. The feature loss has an improvement effect on improving the PSNR and MS-SSIM indicators. Since the influence of the intermediate layer features on the reconstructed image is fully considered, the effect of image reconstruction is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 A flowchart of a deep learning codec network training method provided by an embodiment of the present invention;

[0018] Figure 2 A training principle diagram of a deep learning codec network provided by an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of the structure of a convolution-based auxiliary module provided in an embodiment of the present invention;

[0020] Figure 4 A schematic diagram of a deep learning codec network training system provided by an embodiment of the present invention;

[0021] Figure 5 A schematic diagram of a processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0023] First, the terms that may be used in this article are explained as follows:

[0024] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0025] The following is a detailed description of a training method and device for a deep learning codec network provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professional and technical personnel in the field. If the specific conditions are not specified in the embodiments of the present invention, they are carried out according to the conventional conditions in the field or the conditions recommended by the manufacturer. The reagents or instruments used in the embodiments of the present invention that do not indicate the manufacturer are all conventional products that can be purchased commercially.

[0026] Embodiment 1

[0027] An embodiment of the present invention provides a training method for a deep learning codec network. The deep learning codec network to be trained mainly includes: an encoding part, a super prior probability model and a decoding part. The specific structures involved in each part of the network can refer to conventional technologies and will not be elaborated in the present invention.

[0028] In the embodiment of the present invention, the training of the deep learning codec network includes two stages.

[0029] The first stage: setting multiple convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network;

[0030] The second stage: remove the convolution module and all convolution-based auxiliary modules, input the original image into the deep learning codec network, and calculate the rate-distortion loss as the loss function of the second stage to train the deep learning codec network trained in the first stage.

[0031] The above two stages both involve rate-distortion loss. The following is a brief description of the calculation process of rate-distortion loss: the original image is input into the deep learning codec network, the image features of the original image are obtained through the encoding part, the image features are losslessly entropy encoded and losslessly entropy decoded respectively through the two branches in the super prior probability model, and the image features are deconvolved and nonlinearly transformed through the decoding part to generate a decoded reconstructed image (the output of the decoding part); the rate loss is calculated using the bit rate estimation results obtained by the two branches, and the distortion loss is calculated using the decoded reconstructed image and the original image. The rate loss and distortion loss are collectively referred to as rate-distortion loss.

[0032] The core of the present invention is to introduce a removable convolution-based auxiliary module and convolution module, which are applied in the first stage and removed in the second stage. After the training is completed, the convolution-based auxiliary module and convolution module are not needed, that is, the convolution-based auxiliary module and convolution module will not participate in the subsequent image encoding and decoding process. In the first stage, the output of the convolution-based auxiliary module and the convolution module is used to calculate the feature loss, and the above-mentioned rate-distortion loss is combined as the loss function of the first stage for training. In the second stage, there is no need to calculate the feature loss, and the rate-distortion loss is directly used for training based on the training in the first stage.

[0033] Typically, a threshold of the number of iterations can be set to ensure sufficient training in the first stage, that is, the second stage can be entered after the number of iterations in the first stage reaches the threshold of the number of iterations. Exemplarily, the threshold of the number of iterations can be set to 1.5 million, that is, after 1.5 million iterations in the first stage, the second stage can be entered for training fine-tuning.

[0034] like Figure 2 As shown in Figure 1, it is a training principle diagram of a deep learning codec network, which gives an example of setting up four convolution-based auxiliary modules. Figure 2 Module M is used to represent that the input of the first module M is connected to the input of the first deconvolution layer, and the inputs of the second module M, the third module M, and the fourth module M are connected to the outputs of the first deconvolution layer, the second, and the third deconvolution layer respectively; the output of the decoding network is the decoded reconstructed image, which is processed by a convolution module and used together with the outputs of the four modules M for the calculation of feature loss.

[0035] In the embodiment of the present invention, all the auxiliary modules based on convolution have the same structure; a single auxiliary module based on convolution includes: a first convolution layer, an activation layer and a second convolution layer arranged in sequence. Figure 3 As shown, the structure of a single convolution-based auxiliary module.

[0036] Figure 2-3 In the figure: Resblock3*3 represents a residual block of size 3*3; Conv represents convolution, Deconv represents deconvolution, and the back information represents the related parameters of convolution or deconvolution. Structurally, it is represented as: x*x, y, Sz, activation; where: (1) x*x (for example, 5*5) represents the size of the convolution kernel; (2) y (for example, 192) represents the number of output channels; (3) Sz (for example, S1) can be understood as downsampling the input features. The number z in Sz (for example, 1) represents the sliding step of the convolution kernel. If Sz is omitted, it means that the length and width are consistent with the input; (4) activation represents the specific activation function used. The GDN (generalized divisor normalization) and leaky-relu in the figure represent two types of activation functions. Of course Figure 2-3 The parameter values ​​shown are only examples and do not constitute limitations. In actual applications, users can uniformly set the values ​​of various parameters based on actual conditions or experience.

[0037] In the embodiment of the present invention, the feature loss is calculated using the features output by all the convolution-based auxiliary modules and the features output by the convolution module. The main calculation process includes:

[0038] (1) The features output by all convolution-based auxiliary modules are recorded as the feature set {x1, x2, ..., x n}, where xi represents the features output by the i-th convolution-based auxiliary module, i = 1, 2, ..., n, and n represents the number of convolution-based auxiliary modules.

[0039] (2) The features output by the convolution module are recorded as Features According to the feature set {x1,x2,...,x n} downsample the size of each feature to obtain the downsampled feature set Among them, the down-sampled features The feature x output by the i-th convolution-based auxiliary module i are the same size.

[0040] (3) Using the feature set {x1,x2,...,x n} and downsampled feature set Calculate the feature loss, expressed as:

[0041]

[0042] Among them, L-LOSS represents feature loss and MSE represents mean square error.

[0043] by Figure 2 For example, the number of convolution-based auxiliary modules is 4, and the feature dimensions are: the first one is H / 16, W / 16, C; the second one is H / 8, W / 8, C; the third one is H / 4, W / 4, C; the fourth one is H / 2, W / 2, C; where C is the number of feature channels, and H and W represent the height and width of the original image respectively; after the four features are subjected to a 3*3 convolution, leaky-relu and 1*1 convolution, the number of channels in the feature dimension is 3 (C=3), and the height and width remain unchanged. The features output by the four convolution-based auxiliary modules are defined as x1, x2, x3, and x4 respectively. The reconstructed image is convoluted downsampled four times with different ratios to obtain The sizes correspond one to one, and the mean square error between the downsampled image and the feature map is calculated and averaged:

[0044]

[0045] Combining the above feature loss with the rate-distortion loss, the loss function of the first stage is constructed, which is expressed as:

[0046] RD-LOSS1=R-LOSS+λ1D-LOSS+λ2L-LOSS

[0047] Among them, RD-LOSS1 represents the loss function of the first stage, L-LOSS represents the feature loss; R-LOSS and D-LOSS represent the rate loss and distortion loss respectively, which constitute the rate-distortion loss; λ1 and λ2 represent the hyperparameters for adjusting the proportion of distortion loss and feature loss respectively. In general, λ1 takes multiple code points such as 128, 256, 512, 1024, and λ2 takes 0.01, 0.005, etc. The meaning of the code point is to balance entropy and reconstruction quality. The higher the code point of λ1, the higher the quality of the reconstructed image. The higher λ2 is, the more attention is paid to the intermediate layer of the decoding network.

[0048] The loss function of the second stage is: RD-LOSS2 = R-LOSS + λ1D-LOSS. It should be noted that the symbols shown in the RD-LOSS1 and RD-LOSS2 formulas are only used to indicate the type of corresponding loss, and do not mean that the loss values ​​are the same. For example, the losses in both stages include rate loss and distortion loss, but obviously, the loss values ​​of the two in different stages are different.

[0049] For ease of understanding, the following is a detailed introduction to the two-stage training process. Figure 2 ,The training process of the first and second stages mainly includes:

[0050] 1. The first stage (training with feature loss):

[0051] (1) Input the original image (dimensions: H, W, 3), enter the encoding part, extract the primary features of the image through convolutional neural network and generalized divisor normalization, and use the resblock module to enhance the features in the middle. (The feature dimensions after extraction are: H / 16, W / 16, C, C is the number of channels of the feature)

[0052] (2) The image feature S output by the encoding part enters the hyper-prior probability model (hyper-prior), which contains two branches. Both branches perform bitrate estimation respectively, and the rate loss (R-OSS) is calculated through the bitrate estimation results (bitstream) of the two branches. In the hyper-prior probability model: hyper analysis represents the hyper-prior encoder, hyper synthesis represents the hyper-prior decoder, ROUND represents the quantization operation, AE and AD represent the entropy encoder and entropy decoder (used to perform lossless entropy encoding and lossless entropy decoding), respectively, and LMM represents the Laplace mixture model. The main principles and workflow of the hyper-prior probability model can refer to conventional technologies and will not be elaborated here.

[0053] (3) Since the super-prior probability model performs lossless entropy encoding and decoding, the output of the left branch AD is the image feature S input to the super-prior probability model. It enters the decoding part and undergoes nonlinear transformation through the deconvolution network to generate a decoded reconstructed image (dimension: H, W, 3).

[0054] (4) Calculate the distortion loss D-LOSS between the decoded reconstructed image and the original image, for example, MSE (mean square error), MAE (mean absolute error), LPIPS (learning-based perceptual image block similarity), GAN loss (generative adversarial network loss), etc. Calculate the features obtained by each module M in the decoding part and the features after convolution of the reconstructed image to calculate L-LOSS (for example, MSE / MAE), and combine with the rate loss R-LOSS to obtain the loss function RD-LOSS1 of the first stage.

[0055] (5) Back-propagation is performed through the loss function RD-LOSS1 of the first stage to update the parameters of the deep learning codec network.

[0056] 2. Second stage (training without feature loss):

[0057] (1) Remove the M module and the convolution module after reconstructing the image, and remove the calculation of L-LOSS.

[0058] (2) Load the pre-trained weights of the first stage, reduce the learning rate to 1e-5, and use the rate-distortion loss RD-LOSS calculated using R-LOSS and D-LOSS.

[0059] (3) Back-propagation is performed through the rate-distortion loss RD-LOSS to update the parameters of the deep learning codec network.

[0060] The training method provided by the present invention introduces a removable convolution-based auxiliary module for auxiliary training, and re-convolves the deconvolved features of the decoding part to extract the reconstructed intermediate features, and calculates the feature loss with the down-sampled features obtained by convolution with the final reconstructed image. The feature loss has an improvement effect on improving the PSNR and MS-SSIM indicators. Since the influence of the intermediate layer features on the reconstructed image is fully considered, the effect of image reconstruction is also improved.

[0061] Embodiment 2

[0062] The present invention also provides a training device for a deep learning codec network, which is mainly implemented based on the method provided in the above embodiment, such as Figure 4 As shown, the device mainly includes:

[0063] The first training unit is applied to the first stage, which includes: setting a plurality of convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, and setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all the convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network;

[0064] The second training unit is applied to the second stage, and the second stage includes: removing the convolution module and all convolution-based auxiliary modules, inputting the original image into the deep learning codec network, and calculating the rate-distortion loss as the loss function of the second stage to train the deep learning codec network.

[0065] In the embodiment of the present invention, all convolution-based auxiliary modules have the same structure; a single convolution-based auxiliary module includes: a first convolution layer, an activation layer, and a second convolution layer arranged in sequence.

[0066] In the embodiment of the present invention, calculating the feature loss using the features output by all convolution-based auxiliary modules and the features output by the convolution module includes:

[0067] The features output by all convolution-based auxiliary modules are recorded as the feature set {x1, x2, ..., x n}, where x i represents the features output by the i-th convolution-based auxiliary module, i = 1, 2, ..., n, where n represents the number of convolution-based auxiliary modules;

[0068] The features output by the convolution module are recorded as Features According to the feature set {x1,x2,...,x n} downsample the size of each feature to obtain the downsampled feature set Among them, the down-sampled features The feature x output by the i-th convolution-based auxiliary module i are the same size;

[0069] Using the feature set {x1,x2,...,x n} and downsampled feature set Calculate the feature loss, expressed as:

[0070]

[0071] Among them, L-LOSS represents feature loss and MSE represents mean square error.

[0072] In the embodiment of the present invention, the loss function of the first stage is expressed as:

[0073] RD-LOSS1=R-LOSS+λ1D-LOSS+λ2L-LOSS

[0074] Among them, RD-LOSS1 represents the loss function of the first stage, L-LOSS represents the feature loss; R-LOSS and D-LOSS represent the rate loss and distortion loss respectively, and the two constitute the rate-distortion loss; λ1 and λ2 represent the hyperparameters for adjusting the proportion of distortion loss and feature loss respectively.

[0075] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0076] Embodiment 3

[0077] The present invention also provides a processing device, such as Figure 5 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the methods provided in the aforementioned embodiments.

[0078] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0079] In the embodiment of the present invention, the specific types of the memory, input device and output device are not limited; for example:

[0080] The input device may be a touch screen, an image acquisition device, a physical button or a mouse, etc.;

[0081] The output device may be a display terminal;

[0082] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.

[0083] Embodiment 4

[0084] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.

[0085] In the embodiment of the present invention, the readable storage medium is a computer-readable storage medium and can be set in the aforementioned processing device, for example, as a memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk, etc., which can store program codes.

[0086] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A training method for a deep learning codec network, the deep learning codec network comprising: The encoding part, the super prior probability model and the decoding part are characterized by comprising: The first stage: setting multiple convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network; The second stage: remove the convolution module and all convolution-based auxiliary modules, input the original image into the deep learning codec network, and calculate the rate-distortion loss as the loss function of the second stage to train the deep learning codec network trained in the first stage.

2. A deep learning codec network training method according to claim 1, characterized in that: All convolution-based auxiliary modules have the same structure; a single convolution-based auxiliary module includes: a first convolution layer, an activation layer, and a second convolution layer arranged in sequence.

3. The method for training a deep learning codec network according to claim 1, characterized in that: Calculating feature loss using features output by all convolution-based auxiliary modules and features output by the convolution module includes: The features output by all convolution-based auxiliary modules are recorded as the feature set {x1, x2, ..., x n }, where x i represents the features output by the i-th convolution-based auxiliary module, i = 1, 2, ..., n, where n represents the number of convolution-based auxiliary modules; The features output by the convolution module are recorded as Features According to the feature set {x1,x2,...,x n } downsample the size of each feature to obtain the downsampled feature set Among them, the down-sampled features The feature x output by the i-th convolution-based auxiliary module i are the same size; Using the feature set {x1,x2,...,x n } and downsampled feature set Calculate the feature loss, expressed as: Among them, L-LOSS represents feature loss and MSE represents mean square error.

4. A method for training a deep learning codec network according to claim 1 or 3, characterized in that: The loss function of the first stage is expressed as: RD-LOSS1=R-LOSS+λ1D-LOSS+λ2L-LOSS Among them, RD-LOSS1 represents the loss function of the first stage, L-LOSS represents the feature loss; R-LOSS and D-LOSS represent the rate loss and distortion loss respectively, and the two constitute the rate-distortion loss; λ1 and λ2 represent the hyperparameters for adjusting the proportion of distortion loss and feature loss respectively.

5. A training device for a deep learning codec network, the deep learning codec network comprising: The encoding part, the super prior probability model and the decoding part are characterized by comprising: The first training unit is applied to the first stage, which includes: setting a plurality of convolution-based auxiliary modules, wherein the input of one convolution-based auxiliary module is the input of the first deconvolution layer in the decoding part, and the input of each of the remaining convolution-based auxiliary modules is the output of a single deconvolution layer between the first deconvolution layer and the second to last deconvolution layer, and setting a convolution module, whose input is the output of the decoding part; inputting the original image into the deep learning codec network, calculating the rate-distortion loss, and calculating the feature loss using the features output by all the convolution-based auxiliary modules and the features output by the convolution module, and combining the rate-distortion loss and the feature loss to construct the loss function of the first stage to train the deep learning codec network; The second training unit is applied to the second stage, and the second stage includes: removing the convolution module and all convolution-based auxiliary modules, inputting the original image into the deep learning codec network, and calculating the rate-distortion loss as the loss function of the second stage to train the deep learning codec network.

6. A deep learning codec network training device according to claim 5, characterized in that: All convolution-based auxiliary modules have the same structure; a single convolution-based auxiliary module includes: a first convolution layer, an activation layer, and a second convolution layer arranged in sequence.

7. The training device for a deep learning codec network according to claim 5, characterized in that: Calculating feature loss using features output by all convolution-based auxiliary modules and features output by the convolution module includes: The features output by all convolution-based auxiliary modules are recorded as the feature set {x1, x2, ..., x n }, where x i represents the features output by the i-th convolution-based auxiliary module, i = 1, 2, ..., n, where n represents the number of convolution-based auxiliary modules; The features output by the convolution module are recorded as Features According to the feature set {x1,x2,...,x n } downsample the size of each feature to obtain the downsampled feature set Among them, the down-sampled features The feature x output by the i-th convolution-based auxiliary module i are the same size; Using the feature set {x1,x2,...,x n } and downsampled feature set Calculate the feature loss, expressed as: Among them, L-LOSS represents feature loss and MSE represents mean square error.

8. A training device for a deep learning codec network according to claim 5 or 7, characterized in that: The loss function of the first stage is expressed as: RD-LOSS1=R-LOSS+λ1D-LOSS+λ2L-LOSS Among them, RD-LOSS1 represents the loss function of the first stage, L-LOSS represents the feature loss; R-LOSS and D-LOSS represent the rate loss and distortion loss respectively, and the two constitute the rate-distortion loss; λ1 and λ2 represent the hyperparameters for adjusting the proportion of distortion loss and feature loss respectively.

9. A processing device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images