Image encoding and decoding method, device, equipment, medium and computer program product

Abstract

The invention provides an image encoding and decoding method, device and equipment, a medium and a computer program product. The method comprises the following steps: encoding input image data to obtain encoded image data; determining a signal-to-noise ratio of a channel for transmitting the coded image data, and fusing the coded image data and the signal-to-noise ratio to obtain fused image data; decoding the fused image data based on coding and decoding loss to obtain decoded image data; the coding and decoding loss is determined based on the signal-to-noise ratio. According to the method, the signal-to-noise ratio of the coding and decoding process and the signal-to-noise ratio of the transmission channel are fused, so that the balance between the image compression ratio and the transmission noise immunity is realized, the visual quality and the structural integrity after image transmission are optimized, and the efficient compression transmission of the image is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image encoding and decoding method, device, equipment, medium and computer program product. Background Art

[0002] With the advancement of digital transformation, efficient image data encoding and decoding methods have become an urgent direction to be developed. Efficient image data encoding and decoding methods can improve the image compression efficiency and the visual quality of image display. There are many commonly used loss functions in the image data encoding and decoding process, and each has different focuses. Therefore, how to balance the image data encoding and decoding process, improve the efficiency of image data encoding and decoding, and improve the visual quality of image data after encoding and decoding. Summary of the invention

[0003] The present invention provides an image coding and decoding method, device, equipment, medium and computer program product to solve the defects of low efficiency and poor coding and decoding visual quality in existing image data coding and decoding methods, and to improve the image data coding and decoding efficiency and visual effect.

[0004] The present invention provides an image encoding and decoding method, comprising the following steps.

[0005] Encoding the input image data to obtain encoded image data; determining a signal-to-noise ratio of a channel for transmitting the encoded image data, and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data; The fused image data is decoded based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

[0006] According to an image coding and decoding method provided by the present invention, encoding input image data to obtain coded image data comprises: Encoding the input image data based on a target encoder to obtain encoded image data; the target encoder includes a convolutional layer; The parameters of the convolution layer are determined based on the input image data; the parameters of the convolution layer include the number of convolution layers, the number of network channels, the size of the convolution kernel, the step size, the number of padding pixels, and the activation function.

[0007] According to an image encoding and decoding method provided by the present invention, the target encoder further includes a signal-to-noise ratio processing layer; and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data includes: In the signal-to-noise ratio processing layer, the signal-to-noise ratio is converted based on the dimension of the encoded image data to obtain a converted signal-to-noise ratio; The encoded image data is fused with the converted signal-to-noise ratio to obtain fused image data.

[0008] According to an image encoding and decoding method provided by the present invention, the image encoding and decoding method further includes: A coding loss is determined based on the signal-to-noise ratio, the structural similarity score, and the mean square error.

[0009] According to an image encoding and decoding method provided by the present invention, the determining of the encoding and decoding loss based on the signal-to-noise ratio, the structural similarity score and the mean square error comprises: determining a first weight and a second weight based on the signal-to-noise ratio; The coding loss is determined based on a weighted score and a weighted error; the weighted score is determined based on the structural similarity score and the first weight; and the weighted error is determined based on a mean square error and the second weight.

[0010] According to an image encoding and decoding method provided by the present invention, the sum of the first weight and the second weight is a fixed value; When the signal-to-noise ratio shows a decreasing trend, determining that the first weight shows an increasing trend and the second weight shows a decreasing trend; When the signal-to-noise ratio shows an increasing trend, it is determined that the first weight shows a decreasing trend and the second weight shows an increasing trend.

[0011] The present invention also provides an image encoding and decoding device, comprising the following modules: An image data encoding module, used for encoding input image data to obtain encoded image data; a data fusion module, used to determine a signal-to-noise ratio of a channel for transmitting the encoded image data, and fuse the encoded image data with the signal-to-noise ratio to obtain fused image data; The image data decoding module is used to decode the fused image data based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned image encoding and decoding methods when executing the computer program.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image encoding and decoding method described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image encoding and decoding method described above is implemented.

[0015] The image coding and decoding method, apparatus, device, medium and computer program product provided by the present invention first encode the input image data to obtain the encoded image data, then determine the signal-to-noise ratio of the transmission channel of the encoded image data, further fuse the encoded image data with the signal-to-noise ratio to obtain the fused image data, and finally decode the fused image data based on the coding and decoding loss to obtain the decoded image data, wherein the coding and decoding loss is determined based on the signal-to-noise ratio. The present invention achieves a balance between image compression rate and transmission noise resistance by fusing the coding and decoding process with the signal-to-noise ratio of the transmission channel, optimizes the visual quality and structural integrity of the image after transmission, and realizes efficient compression transmission of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flowchart diagrams of the image encoding and decoding method provided by the present invention.

[0018] Figure 2 This is the second flowchart of the image encoding and decoding method provided by the present invention.

[0019] Figure 3 This is the third flowchart of the image encoding and decoding method provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the image encoding and decoding device provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Combine the following Figure 1-Figure 5 The present invention describes the image coding and decoding method, apparatus, device, medium and computer program product.

[0024] Figure 1It is one of the flowcharts of the image encoding and decoding method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 100: Encode the input image data to obtain encoded image data; The image encoding and decoding method provided by this application mainly includes the image encoding process and the image decoding process. Among them, the image encoding and decoding method provided by this application mainly includes the following three core links: 1. Extracting key information of the image through the convolution layer for encoding (the deconvolution layer restores the image by encoding key information); 2. The signal-to-noise ratio (SNR) processing layer integrates the channel state with the image encoding; 3. The loss function is automatically adjusted based on the visual mode.

[0025] The encoder consists of a convolutional layer and an SNR processing layer. It is mainly used to extract the characteristic rules of the input image data and determine the image encoding strategy according to the transmission channel conditions to achieve more anti-noise capabilities at a certain compression level. Taking the images of Kodak24 (a commonly used data set for digital image processing) as an example, the images are all 512×768 pixels and have three RGB color channels.

[0026] First, the number of convolution layers, the number of network channels in each layer, the size of the convolution kernel, the step size, the number of padding pixels, the activation function and other parameters are designed according to the input image. The specific content will be described in detail in the following embodiments.

[0027] Step 200: determine the signal-to-noise ratio of the channel for transmitting the encoded image data, and fuse the encoded image data with the signal-to-noise ratio to obtain fused image data; Specifically, the encoded information (encoded image data) is sent to the decoder through the SNR noisy channel. The decoder consists of a deconvolution layer and an SNR processing layer. The decoder is used to integrate the received encoded information and the channel SNR for decoding to restore the image before encoding, that is, the input image data, as much as possible.

[0028] Step 300: Decode the fused image data based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

[0029] Specifically, the decoding process corresponds to the encoding process. The decoder first decodes the received fused image data through a deconvolution layer, or the deconvolution process can be omitted and the next step can be directly entered. Figure 2 In the example shown, a deconvolution layer is applied to the received fused image data without changing the data size.

[0030] Using the same SNR processing layer as the encoder, the decoder maps the SNR information to the appropriate dimension and acts on the data obtained by the previous deconvolution step. The decoding process in this example is the same as the processing in the encoder.

[0031] Then, the decoder designs parameters such as the number of deconvolution layers and the number of network channels in each layer, convolution kernel size, step size, number of padding pixels, output padding prime number, and activation function to restore the image. Figure 2 In the example shown, a three-layer serial deconvolution network is also designed. The number of input channels and output channels are (64, 32), (32, 16), (16, 3), respectively. The kernel size of each layer is 3, the step size is 2, the number of padded pixels is 1, the number of output padded pixels is 1, and the activation function is the Tanh function. The encoded data (64, 64, 96) is restored to (3, 512, 768) after the SNR processing layer and the deconvolution network decoding. The loss function of the encoding and decoding process will be described in detail below.

[0032] In this embodiment, first, the input image data is encoded to obtain encoded image data, then the signal-to-noise ratio of the transmission channel of the encoded image data is determined, and the encoded image data is further fused with the signal-to-noise ratio to obtain fused image data, and finally the fused image data is decoded based on the coding and decoding loss to obtain decoded image data, wherein the coding and decoding loss is determined based on the signal-to-noise ratio. The present invention achieves a balance between image compression rate and transmission noise resistance by fusing the coding and decoding process with the signal-to-noise ratio of the transmission channel, optimizes the visual quality and structural integrity of the image after transmission, and realizes efficient compression transmission of the image.

[0033] Figure 3 This is the third flow chart of the image encoding and decoding method provided by the present invention, such as Figure 3 As shown, the method may also include: Step 110: Encode the input image data based on a target encoder to obtain encoded image data; the target encoder includes a convolutional layer; Step 120: Determine the parameters of the convolution layer based on the input image data; the parameters of the convolution layer include the number of convolution layers, the number of network channels, the size of the convolution kernel, the step size, the number of padding pixels, and the activation function.

[0034] like Figure 2 As shown in the example, the encoder designs a three-layer series convolutional network. The input layer and output channels of each convolutional network are (3, 16), (16, 32), (32, 64), the kernel size of each layer is 3, the step size is 2, the number of padded pixels is 1, and the activation function is the Tanh function. The original image (3, 512, 768) becomes (64, 64, 96) after the convolutional network extracts features, which is compressed to one third of the original.

[0035] In the SNR processing layer of the encoder, the signal-to-noise ratio value of the transmission channel is converted into a format that meets the dimensionality requirements and used for image feature data. Figure 2In the example, a linear layer is used to map the signal-to-noise ratio value from 1 to 10 values, which are then mapped to 64 values ​​after the PRelu activation function, and then multiplied to each image feature. The size of the calculated image feature data remains unchanged. After that, a convolutional layer or SNR processing layer can be added to the encoder according to the actual situation for further compression or encoding. Figure 2 In the example shown, a convolutional layer is used without changing the data size to more fully integrate the channel SNR information.

[0036] This embodiment achieves a balance between image compression rate and transmission noise resistance by setting a convolution layer and an SNR processing layer in the encoder to fuse image data with channel signal-to-noise ratio information.

[0037] In one embodiment, the image encoding and decoding method provided in the embodiment of the present application may further include: Step 210: at the signal-to-noise ratio processing layer, transform the signal-to-noise ratio based on the dimension of the encoded image data to obtain a transformed signal-to-noise ratio; Step 220: Fuse the encoded image data with the converted signal-to-noise ratio to obtain fused image data.

[0038] The process of fusing the signal-to-noise ratio value with the input image data is as follows: In the SNR processing layer of the encoder, the signal-to-noise ratio value of the channel is first converted into a format that meets the image dimension requirements, and the signal-to-noise ratio value is applied to the image feature data. Figure 2 In the example shown, the signal-to-noise ratio value is mapped to 10 values ​​through a linear layer, and then mapped to 64 values ​​after the activation function. Then, the mapped signal-to-noise ratio value is applied to each image feature through a certain algorithm (such as multiplication) to achieve the fusion of the signal-to-noise ratio value and the input image data.

[0039] This embodiment achieves a balance between image compression rate and transmission noise resistance by fusing the signal-to-noise ratio value with the input image data.

[0040] In one embodiment, the image encoding and decoding method provided in the embodiment of the present application may further include: Step 400: Determine the encoding and decoding loss based on the signal-to-noise ratio, the structural similarity score, and the mean square error.

[0041] The image encoding and decoding method provided in the embodiment of the present application may further include: Step 410: determining a first weight and a second weight based on the signal-to-noise ratio; Step 420: Determine the encoding and decoding loss based on the weighted score and the weighted error; the weighted score is determined based on the structural similarity score and the first weight; the weighted error is determined based on the mean square error and the second weight.

[0042] In the process of fusing SNR and image data, more possible SNR values ​​need to be trained. More SNR value data can be used to train the fusion process and encoding and decoding process. For example, using AWGN additive white Gaussian noise channel, the SNR value used for training is an integer sample randomly drawn from a normal distribution with a mean of 10 and a variance of 5 in the set [0, 20].

[0043] When the channel environment is bad and the signal-to-noise ratio is low, the transmitted image will usually be blurred and pixelated, which will cause quality degradation. In this case, the details of the image are difficult to discern, and the human eye will prioritize the overall structural similarity of the image rather than the image details during visual processing. Therefore, when designing the loss function of the (image encoding and decoding) deep learning model, the overall structural similarity score of the image (such as SSIM) can be emphasized, which helps optimize the model to maintain the overall visual and structural integrity of the image, as shown in Formula 1.

[0044] ; (1)

[0045] in, and represent the average values ​​of x and y respectively; and denote the covariance of x and y respectively; and It represents a constant that maintains stability. Therefore, for the SSIM of the entire image, we can divide the image into many small windows for calculation, and then calculate the SSIM in each window and find the average value.

[0046] When the channel conditions are good and the signal-to-noise ratio is high, the image transmission quality is high and the image detail loss is small. In this case, the optimization space of SSIM is limited, and even a slight improvement is difficult to bring significant visual improvement. In addition, as the SNR increases, the impact of SSIM in improving the overall image quality gradually decreases. In this case, it is more appropriate to use the mean square error MSE, because the calculation characteristics of MSE are simple and fast, and it can provide accurate feedback on the image detail level. This makes MSE particularly useful in high signal-to-noise ratio environments, and can accurately evaluate the absolute error of the image, so as to better understand and optimize the image processing algorithm, as shown in Formula 2.

[0047] ; (2)

[0048] Among them, H and W represent the height and width of the image respectively. Indicates that the original image is at position The pixel value of Represents the reconstructed image at position The pixel value of .

[0049] Based on the above content, the present invention proposes a loss function determined based on the combination of MSE and SSIM, and gives an indicator weight that is more in line with human visual logic according to the channel SNR characteristics.

[0050] This embodiment provides a solution for determining the loss function of a coded or decoded image that is more in line with human visual logic through a combination of loss functions.

[0051] In one embodiment, the image encoding and decoding method provided in the embodiment of the present application may further include: Step 411: When the signal-to-noise ratio is on a downward trend, determine that the first weight is on an upward trend and the second weight is on a downward trend; Step 412: When the signal-to-noise ratio shows an increasing trend, determine that the first weight shows a decreasing trend and the second weight shows an increasing trend.

[0052] As shown in Formula 3, Formula 3 is a codec image loss function LOSS provided by the present invention that is more in line with human visual logic, wherein, , when SNR belongs to different value ranges, The value of is different. In formula 3, That is, the second weight in this embodiment, is the first weight in this embodiment.

[0053] ; (3)

[0054] When SNR is in the range of 0-5, the value of the first weight can be 0.6 and the value of the second weight can be 0.4; when SNR is in the range of 5-10, the value of the first weight can be 0.3 and the value of the second weight can be 0.7; when SNR is in the range of 10-20, the value of the first weight can be 0.1 and the value of the second weight can be 0.9. The optimizer and learning rate used by the model are not required. In this example, the Adma optimizer and the learning rate of 0.001 are used, and the loss converges after 10,000 trainings.

[0055] This embodiment proposes an image coding and decoding solution that is more in line with human visual logic by combining the loss functions and flexibly adjusting the loss function of the image coding and decoding process in combination with the channel signal-to-noise ratio value.

[0056] The derivation process of the beneficial effects that can be achieved by the image encoding and decoding method provided by the present invention is as follows: The present invention designs an image data encoding and decoding method that is adaptive to channel conditions through a deep learning network model, and improves data characterization capabilities by mining deep data rules. At the same time, based on human visual perception, an adaptive loss function combining MSE and SSIM is proposed. The algorithm dynamically adjusts the encoding and decoding scheme and the weights of the two loss functions according to the channel SNR to achieve a balance between image compression rate and image noise resistance, optimize the visual quality and structural integrity in image transmission, and realize efficient image transmission.

[0057] The specific manifestations are: 1. The present invention realizes the extraction of key information of image data through the convolution layer to achieve the effect of image compression, thereby reducing the amount of image data transmission. In the above embodiment, the size of the image data is 3×512×768, and the size of the data to be transmitted after encoding is 64×64×96, which is compressed to one-third of the original image. Using different deep learning network models and parameters can also achieve better compression effects. This application greatly reduces the bandwidth required for data transmission, increases the speed of image data transmission, reduces the delay and cost of image data transmission, and achieves the purpose of optimizing resource utilization.

[0058] 2. The present invention adopts lossy compression, which aims to reduce the compression ratio while maintaining the integrity and accuracy of the image information after decoding through the noisy channel as much as possible. Through the design and arrangement of the network model and the SNR processing layer, the channel conditions are fully considered while encoding, ensuring that the data encoded according to the channel conditions has a small deviation from the original data in terms of the overall numerical range, distribution characteristics and change trends after decoding based on the channel conditions, thereby achieving maximum fidelity. In the above embodiment, the peak signal-to-noise ratio PSNR (the larger the value, the smaller the distortion), the learning perceptual image block similarity LPIPS (the smaller the value, the more similar) and the multi-scale structural similarity MS-SSIM (the larger the value, the closer to the original image) are first used to compare the difference between the effects of uncoded image transmission and coded image transmission (model training can be performed using only MSE as the loss function). By selecting different models and parameter configurations, the degree of restoration after image encoding and decoding can also be further optimized.

[0059] 3. The loss function based on visual mode proposed in this invention can automatically adjust the training loss focus under different SNRs, give priority to meeting the main visual requirements, and give the optimal visual decoded image under the conditions. The effects of training models using weighted loss function and single loss function MSE and SSIM are compared.

[0060] 4. Different network models, training parameters, SNR processing layer position arrangement and loss function parameters Each value has its applicable scenario and advantages and disadvantages. For different transmission scenarios and transmission requirements, selecting appropriate network models and training parameters, and reasonably arranging the positions and dimensions of convolution layers (and deconvolution layers) and SNR processing layers depends on the degree of information loss acceptable to both parties of the transmission, network conditions, compression ratio requirements, decompression speed, cost, and performance. The present invention allows the adjustment of the number and position of multiple network models and SNR processing layers, which is more flexible.

[0061] 5. The encoding process also realizes protocol encryption of data to a certain extent, enhances the privacy and security of data, makes it more difficult to interpret and steal during transmission, reduces the loss of data directly exposed to attacks in an unsafe environment, and thus protects the confidentiality of data.

[0062] The image coding and decoding device provided by the present invention is described below. The image coding and decoding device described below and the image coding and decoding method described above can be referred to each other.

[0063] Please refer to Figure 4 The present invention also provides an image encoding and decoding device, comprising: The image data encoding module 401 is used to encode the input image data to obtain encoded image data; A data fusion module 402, configured to determine a signal-to-noise ratio of a channel for transmitting the encoded image data, and fuse the encoded image data with the signal-to-noise ratio to obtain fused image data; The image data decoding module 403 is used to decode the fused image data based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

[0064] Optionally, the image data encoding module includes: An image data encoding unit, configured to encode input image data based on a target encoder to obtain encoded image data; the target encoder comprises a convolutional layer; The encoding convolution layer parameter determination unit is used to determine the parameters of the convolution layer based on the input image data; the parameters of the convolution layer include the number of convolution layers, the number of network channels, the convolution kernel size, the step size, the number of padding pixels and the activation function.

[0065] Optionally, the target encoder further includes a signal-to-noise ratio processing layer; and the data fusion module includes: A signal-to-noise ratio conversion unit, configured to convert the signal-to-noise ratio at the signal-to-noise ratio processing layer based on the dimension of the encoded image data to obtain a converted signal-to-noise ratio; A data fusion unit is used to fuse the encoded image data with the converted signal-to-noise ratio to obtain fused image data.

[0066] Optionally, the image encoding and decoding device further includes: The coding loss determination module is used to determine the coding loss based on the signal-to-noise ratio, the structural similarity score and the mean square error.

[0067] Optionally, the determining the encoding and decoding loss based on the signal-to-noise ratio, the structural similarity score, and the mean square error includes: a weight determination unit, configured to determine a first weight and a second weight based on the signal-to-noise ratio; A codec loss determination unit is used to determine the codec loss based on a weighted score and a weighted error; the weighted score is determined based on the structural similarity score and the first weight; the weighted error is determined based on a mean square error and the second weight.

[0068] Optionally, the sum of the first weight and the second weight is a fixed value; and determining the first weight and the second weight based on the signal-to-noise ratio includes: A first weight change trend determining unit, configured to determine, when the signal-to-noise ratio shows a decreasing trend, that the first weight shows an increasing trend and the second weight shows a decreasing trend; The second weight change trend determining unit is used to determine, when the signal-to-noise ratio shows an increasing trend, that the first weight shows a decreasing trend and the second weight shows an increasing trend.

[0069] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the image coding and decoding method, which includes: encoding the input image data to obtain coded image data; determining the signal-to-noise ratio of the channel for transmitting the coded image data, fusing the coded image data with the signal-to-noise ratio to obtain fused image data; decoding the fused image data based on the coding and decoding loss to obtain decoded image data; the coding and decoding loss is determined based on the signal-to-noise ratio.

[0070] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0071] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image encoding and decoding methods provided by the above methods, which include: encoding input image data to obtain encoded image data; determining the signal-to-noise ratio of a channel for transmitting the encoded image data, and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data; decoding the fused image data based on a coding and decoding loss to obtain decoded image data; the coding and decoding loss is determined based on the signal-to-noise ratio.

[0072] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image encoding and decoding methods provided by the above-mentioned methods, the method comprising: encoding input image data to obtain encoded image data; determining a signal-to-noise ratio of a channel for transmitting the encoded image data, and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data; decoding the fused image data based on a coding and decoding loss to obtain decoded image data; the coding and decoding loss is determined based on the signal-to-noise ratio.

[0073] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image encoding and decoding method, characterized in that: include: Encoding the input image data to obtain encoded image data; determining a signal-to-noise ratio of a channel for transmitting the encoded image data, and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data; The fused image data is decoded based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

2. The image encoding and decoding method according to claim 1, characterized in that: The encoding of the input image data to obtain the encoded image data comprises: Encoding the input image data based on a target encoder to obtain encoded image data; the target encoder includes a convolutional layer; The parameters of the convolution layer are determined based on the input image data; the parameters of the convolution layer include the number of convolution layers, the number of network channels, the size of the convolution kernel, the step size, the number of padding pixels, and the activation function.

3. The image encoding and decoding method according to claim 1, characterized in that: The target encoder further includes a signal-to-noise ratio processing layer; and fusing the encoded image data with the signal-to-noise ratio to obtain fused image data includes: In the signal-to-noise ratio processing layer, the signal-to-noise ratio is converted based on the dimension of the encoded image data to obtain a converted signal-to-noise ratio; The encoded image data is fused with the converted signal-to-noise ratio to obtain fused image data.

4. The image encoding and decoding method according to claim 1, characterized in that: The image encoding and decoding method further includes: A coding loss is determined based on the signal-to-noise ratio, the structural similarity score, and the mean square error.

5. The image encoding and decoding method according to claim 4, characterized in that: The determining of the encoding and decoding loss based on the signal-to-noise ratio, the structural similarity score, and the mean square error includes: determining a first weight and a second weight based on the signal-to-noise ratio; The coding loss is determined based on a weighted score and a weighted error; the weighted score is determined based on the structural similarity score and the first weight; and the weighted error is determined based on a mean square error and the second weight.

6. The image encoding and decoding method according to claim 5, characterized in that: The sum of the first weight and the second weight is a fixed value; and determining the first weight and the second weight based on the signal-to-noise ratio includes: When the signal-to-noise ratio shows a decreasing trend, determining that the first weight shows an increasing trend and the second weight shows a decreasing trend; When the signal-to-noise ratio shows an increasing trend, it is determined that the first weight shows a decreasing trend and the second weight shows an increasing trend.

7. An image encoding and decoding device, characterized in that: include: An image data encoding module, used for encoding input image data to obtain encoded image data; a data fusion module, used to determine a signal-to-noise ratio of a channel for transmitting the encoded image data, and fuse the encoded image data with the signal-to-noise ratio to obtain fused image data; The image data decoding module is used to decode the fused image data based on a coding loss to obtain decoded image data; the coding loss is determined based on the signal-to-noise ratio.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image encoding and decoding method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image encoding and decoding method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image encoding and decoding method according to any one of claims 1 to 6 is implemented.

Images