Distributed decoding method and related device

Abstract

The embodiment of the invention provides a distributed decoding method and a related device, and the method comprises the steps: receiving a bit stream which comprises an encoding result of gray information of an input image; reconstructing gray information of the input image according to the bit stream; determining an optical flow between the input image and a related image according to the reconstructed gray information of the input image and the related image, wherein the related image is an image associated with the input image; sampling the related image through the optical flow to obtain a sampling image; and reconstructing the input image according to the gray scale information of the sampling image and the reconstructed input image. According to the embodiment of the invention, the reconstruction quality of the input image can be improved.

Description

Technical Field

[0001] The present application relates to the field of video encoding technology, and in particular to a distributed decoding method and related devices. Background Art

[0002] Distributed source coding is a key area in information theory that focuses on how to efficiently compress data when two or more related sources are independently coded. The concept was first proposed by Slepian and Wolf in 1976, who showed that, ideally, even if the sources are coded independently, the same compression rate can be achieved as joint coding. However, despite the theoretical equivalence of the achievable compression rates with or without side information, in practice there is still a gap in the compression rates of the two mechanisms. In 2020, Ayzik et al. proposed a distributed source compression algorithm DSIN (Ayzik S, Avidan S. Deep image compression using decoder side information [C] / / Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16. Springer International Publishing, 2020: 699-714.) using side information. The encoder and decoder can be any autoencoder architecture. The core of this method is the SI-Finder module. Referring to the block matching enhancement method, the rough reconstruction result of the input image x is divided into several small blocks, and the same encoding and decoding process is performed on the related image y. Each small block of the input image x is matched in the reconstruction result of the related image y, and the corresponding small block in the input image x is enhanced by the corresponding position of the most matching block in the original image of the related image y.

[0003] The problem with the prior art is that the input image x and the related image y have some local deformation due to the deviation of the viewing angle. Therefore, the DSIN directly uses the related image y to perform image enhancement, and the result is too rough and the effect is not good. Summary of the invention

[0004] The embodiment of the present application discloses a distributed decoding method and related devices, which can improve the reconstruction quality of input images.

[0005] In a first aspect, an embodiment of the present application provides a distributed decoding method, the method comprising:

[0006] Receiving a bit stream, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0007] Reconstructing grayscale information of the input image according to the bit stream;

[0008] Determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image;

[0009] Sampling the related image by the optical flow to obtain a sampled image;

[0010] The input image is reconstructed according to the sampling image and the grayscale information of the reconstructed input image.

[0011] In the above method, the optical flow is calculated according to the grayscale information of the input image and the related image, and then the input image is aligned according to the optical flow to obtain a sampled image. Since the sampled image is an aligned image, that is, aligned side information, the input image is reconstructed based on the sampled image and the grayscale information, which can more accurately and fully utilize the side information, thereby further improving the reconstruction quality.

[0012] In combination with the first aspect, in a possible implementation manner of the first aspect, the present invention further includes:

[0013] Extracting common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics;

[0014] The step of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image comprises:

[0015] The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

[0016] In this way, the public information is also used as input data for reconstructing the input image, which can make up for the deficiencies of the sampled image and grayscale information and enhance the reconstruction effect of the input image.

[0017] In combination with the first aspect, or any of the foregoing possible implementations of the first aspect, in another possible implementation of the first aspect, before extracting the common information between the sampled image and the related image, the step further includes:

[0018] It is determined that the compression ratio of the encoding device is higher than a preset threshold.

[0019] It can be understood that when the compression ratio is higher than the preset threshold, the grayscale information has a limited effect in the reconstruction of the input image. Therefore, public information is introduced to reconstruct the input image only when the compression ratio is higher than the preset threshold, otherwise public information is not introduced. Therefore, this method can not only ensure the reconstruction effect of the input image, but also minimize the amount of calculation.

[0020] In combination with the first aspect, or any of the foregoing possible implementations of the first aspect, in yet another possible implementation of the first aspect, the following further includes:

[0021] Generating a first missing pattern corresponding to the sampled image according to the optical flow, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0022] Sending first indication information to an encoding device, where the first indication information is used to indicate the first missing pattern;

[0023] receiving the image encoding information of the target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0024] The input image is reconstructed according to the image encoding information of the target position.

[0025] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0026] In combination with the first aspect, or any of the foregoing possible implementations of the first aspect, in another possible implementation of the first aspect, before generating the first missing pattern corresponding to the sampled image according to the optical flow, the step further includes:

[0027] It is determined that the reconstruction result of the input image does not meet a preset condition.

[0028] In this implementation, the missing part of the information is fed back to the encoding device only when the reconstruction result of the input image does not meet the preset conditions. That is, the input image is reconstructed again only when the reconstruction result does not meet the preset conditions, instead of reconstructing it again under any circumstances, thus saving computing overhead.

[0029] In combination with the first aspect, or any one of the foregoing possible implementations of the first aspect, in another possible implementation of the first aspect, reconstructing the input image according to the image encoding information of the target position includes:

[0030] Completing the sampled image according to the image coding information of the target position;

[0031] The input image is reconstructed according to the completed sampling image and the grayscale information of the reconstructed input image.

[0032] In this method, the image encoding information at the target position is used to complete the sampled image. After the completion is completed, the previous reconstruction operation based on the sampled image and grayscale information needs to be repeated.

[0033] In combination with the first aspect, or any of the foregoing possible implementations of the first aspect, in another possible implementation of the first aspect, reconstructing the input image according to the encoded information of the target position includes:

[0034] The reconstructed input image is reconstructed again according to the encoding information of the target position to obtain a new input image.

[0035] In this implementation, the encoded information of the target position can be used to directly enhance the previously reconstructed input image without completing the sampled image.

[0036] In combination with the first aspect, or any one of the above possible implementations of the first aspect, in another possible implementation of the first aspect, the image encoding information of the target position includes residual information of the grayscale information of the image of the target position relative to the image of the target position.

[0037] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0038] In combination with the first aspect, or any one of the foregoing possible implementations of the first aspect, in another possible implementation of the first aspect, the first indication information includes the first missing pattern, that is, missing values ​​of the image that need to be sent.

[0039] In combination with the first aspect, or any of the above possible implementations of the first aspect, in another possible implementation of the first aspect, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to a respective number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0040] In combination with the first aspect, or any of the foregoing possible implementations of the first aspect, in yet another possible implementation of the first aspect, the following further includes:

[0041] Update the missing pattern position library according to the positions where information is missing in the image within a preset time period in the past;

[0042] The updated missing pattern library is sent to the encoding device.

[0043] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0044] In a second aspect, an embodiment of the present application provides a distributed decoding method, the method comprising:

[0045] Generate a first missing pattern corresponding to the sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0046] Sending first indication information to an encoding device, where the first indication information is used to indicate the first missing pattern;

[0047] receiving image encoding information of a target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0048] The input image is reconstructed according to the image encoding information of the target position.

[0049] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0050] In combination with the second aspect, in a possible implementation manner of the second aspect, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0051] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0052] In combination with the second aspect, or any one of the foregoing possible implementations of the second aspect, in yet another possible implementation of the second aspect, the first indication information includes a missing value for the first missing pattern, that is, an image that needs to be sent.

[0053] In combination with the second aspect, or any of the above possible implementations of the second aspect, in another possible implementation of the second aspect, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern position library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0054] In combination with the second aspect, or any of the foregoing possible implementations of the second aspect, in yet another possible implementation of the second aspect, the following further includes:

[0055] Update the missing pattern library according to the positions where information is missing in the images within a preset time period in the past;

[0056] The updated missing pattern library is sent to the encoding device.

[0057] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0058] In a third aspect, an embodiment of the present application provides a distributed encoding method, the method comprising:

[0059] Sending a bit stream to a decoding device, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0060] receiving first indication information sent by a decoding device, wherein the first indication information is used to indicate a first missing pattern, the first missing pattern marks whether there is information missing at each position divided on a sampled image, the sampled image being an image obtained according to the bit stream and a related image of the input image;

[0061] The image coding information of the target position is sent to the decoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing.

[0062] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0063] In combination with the third aspect, in a possible implementation manner of the third aspect, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0064] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0065] In combination with the third aspect, or any one of the foregoing possible implementations of the third aspect, in yet another possible implementation of the third aspect, the first indication information includes the first missing pattern, that is, missing values ​​of the image that need to be sent.

[0066] In combination with the third aspect, or any of the above possible implementations of the third aspect, in another possible implementation of the third aspect, the method is applied to a coding device, the coding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0067] In combination with the third aspect, or any of the foregoing possible implementations of the third aspect, in yet another possible implementation of the third aspect, before sending the image encoding information of the first missing pattern target position to the decoding device, the step further includes:

[0068] Determine the target location according to the identifier in the first indication information;

[0069] An image of the target location is encoded.

[0070] In combination with the third aspect, or any of the foregoing possible implementations of the third aspect, in yet another possible implementation of the third aspect, the following further includes:

[0071] The updated missing pattern library sent by the receiving decoding device is received.

[0072] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0073] In a fourth aspect, an embodiment of the present application provides a distributed decoding apparatus, which may be a decoding device or a device or module in a decoding device, and includes:

[0074] A receiving module, configured to receive a bit stream, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0075] A decoding module, used for reconstructing the grayscale information of the input image according to the bit stream;

[0076] A determination module, configured to determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image;

[0077] A sampling module, used for sampling the relevant image through the optical flow to obtain a sampled image;

[0078] An enhancement module is used to reconstruct the input image according to the sampled image and the grayscale information of the reconstructed input image.

[0079] In the above method, the optical flow is calculated according to the grayscale information of the input image and the related image, and then the input image is aligned according to the optical flow to obtain a sampled image. Since the sampled image is an aligned image, that is, aligned side information, the input image is reconstructed based on the sampled image and the grayscale information, which can more accurately and fully utilize the side information, thereby further improving the reconstruction quality.

[0080] In combination with the fourth aspect, in a possible implementation manner of the fourth aspect, the device further includes:

[0081] A common information extraction module, used to extract common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics;

[0082] In terms of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image, the enhancement module is specifically used for:

[0083] The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

[0084] In this way, the public information is also used as input data for reconstructing the input image, which can make up for the deficiencies of the sampled image and grayscale information and enhance the reconstruction effect of the input image.

[0085] In combination with the fourth aspect, or any one of the above-mentioned possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, a judgment module is also included, which is used to determine that the compression ratio of the encoding device is higher than a preset threshold before extracting the common information between the sampled image and the related image.

[0086] It can be understood that when the compression ratio is higher than the preset threshold, the grayscale information has a limited effect in the reconstruction of the input image. Therefore, public information is introduced to reconstruct the input image only when the compression ratio is higher than the preset threshold, otherwise public information is not introduced. Therefore, this method can not only ensure the reconstruction effect of the input image, but also minimize the amount of calculation.

[0087] In combination with the fourth aspect, or any of the foregoing possible implementations of the fourth aspect, in yet another possible implementation of the fourth aspect, the following further includes:

[0088] A missing pattern generating module, used to generate a first missing pattern corresponding to the sampled image according to the optical flow, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0089] a missing pattern feedback module, configured to send first indication information to the encoding device, where the first indication information is used to indicate the first missing pattern;

[0090] a residual decoding module, configured to receive the image encoding information of the target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0091] The enhancement module is further used to reconstruct the input image according to the image coding information of the target position.

[0092] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0093] In combination with the fourth aspect, or any one of the above possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, it also includes: an analysis module, used to determine that the reconstruction result of the input image does not meet a preset condition before generating the first missing pattern corresponding to the sampled image according to the optical flow.

[0094] In this implementation, the missing part of the information is fed back to the encoding device only when the reconstruction result of the input image does not meet the preset conditions. That is, the input image is reconstructed again only when the reconstruction result does not meet the preset conditions, instead of reconstructing it again under any circumstances, thus saving computing overhead.

[0095] In combination with the fourth aspect, or any one of the foregoing possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, in the aspect of reconstructing the input image according to the image coding information of the target position, first, a missing pattern completion module completes the sampled image according to the image coding information of the target position;

[0096] The enhancement module is specifically used to reconstruct the input image according to the completed sampling image and the grayscale information of the reconstructed input image.

[0097] In this method, the image encoding information at the target position is used to complete the sampled image. After the completion is completed, the previous reconstruction operation based on the sampled image and grayscale information needs to be repeated.

[0098] In combination with the fourth aspect, or any of the foregoing possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, in the aspect of reconstructing the input image according to the encoded information of the target position, the enhancement module is specifically used to:

[0099] The reconstructed input image is reconstructed again according to the encoding information of the target position to obtain a new input image.

[0100] In this implementation, the encoded information of the target position can be used to directly enhance the previously reconstructed input image without completing the sampled image.

[0101] In combination with the fourth aspect, or any one of the above-mentioned possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, the image encoding information of the target position includes residual information of the grayscale information of the image of the target position relative to the image of the target position.

[0102] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0103] In combination with the fourth aspect, or any one of the foregoing possible implementations of the fourth aspect, in yet another possible implementation of the fourth aspect, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0104] In combination with the fourth aspect, or any of the above possible implementations of the fourth aspect, in another possible implementation of the fourth aspect, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0105] In combination with the fourth aspect, or any of the foregoing possible implementations of the fourth aspect, in yet another possible implementation of the fourth aspect, the apparatus further includes:

[0106] An updating module, used for updating the missing pattern position library according to the positions where information is missing in the image within a preset time period in the past;

[0107] A synchronization module is used to send the updated missing pattern library to the encoding device.

[0108] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0109] In a fifth aspect, an embodiment of the present application provides a distributed decoding apparatus, which may be a decoding device or a device or module in a decoding device, and includes:

[0110] A missing pattern generating module, used to generate a first missing pattern corresponding to the sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0111] a missing pattern feedback module, configured to send first indication information to the encoding device, where the first indication information is used to indicate the first missing pattern;

[0112] a residual decoding module, configured to receive image encoding information of a target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0113] The missing pattern completion module is used to reconstruct the input image according to the image coding information of the target position.

[0114] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0115] In combination with the fifth aspect, in a possible implementation manner of the fifth aspect, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0116] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0117] In combination with the fifth aspect, or any one of the foregoing possible implementations of the fifth aspect, in another possible implementation of the fifth aspect, the first indication information includes a missing value for the first missing pattern, that is, an image that needs to be sent.

[0118] In combination with the fifth aspect, or any of the above possible implementations of the fifth aspect, in another possible implementation of the fifth aspect, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern position library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0119] In combination with the fifth aspect, or any of the foregoing possible implementations of the fifth aspect, in yet another possible implementation of the fifth aspect, the device further includes:

[0120] An updating module, used for updating the missing pattern library according to the positions where information is missing in the images within a preset time period in the past;

[0121] A synchronization module is used to send the updated missing pattern library to the encoding device.

[0122] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0123] In a sixth aspect, an embodiment of the present application provides a distributed encoding device, which may be an encoding device or a device or module in an encoding device, and the device includes:

[0124] A grayscale information encoding module, used to send a bit stream to a decoding device, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0125] a feedback receiving module, configured to receive first indication information sent by a decoding device, wherein the first indication information is used to indicate a first missing pattern, the first missing pattern marks whether there is information missing at each position divided on a sampled image, and the sampled image is an image obtained according to the bit stream and a related image of the input image;

[0126] The residual coding module is used to send the image coding information of the target position to the decoding device, wherein the target position is the position where the existence information of the first missing pattern mark is missing.

[0127] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0128] In combination with the sixth aspect, in a possible implementation manner of the sixth aspect, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0129] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0130] In combination with the sixth aspect, or any one of the foregoing possible implementations of the sixth aspect, in another possible implementation of the sixth aspect, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0131] In combination with the sixth aspect, or any of the above possible implementations of the sixth aspect, in another possible implementation of the sixth aspect, the method is applied to a coding device, the coding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0132] In combination with the sixth aspect, or any of the foregoing possible implementations of the sixth aspect, in another possible implementation of the sixth aspect:

[0133] The feedback receiving module is further used to determine the target position according to the identifier in the first indication information;

[0134] The residual encoding module is also used to encode the image at the target position.

[0135] In combination with the sixth aspect, or any of the foregoing possible implementations of the sixth aspect, in yet another possible implementation of the sixth aspect, the following further includes:

[0136] The synchronization module is used to receive the updated missing pattern library sent by the decoding device.

[0137] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0138] In the seventh aspect, an embodiment of the present application provides a decoding device, which includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call the computer program in the memory to implement the method described in the first aspect, or any possible implementation of the first aspect, or the second aspect, or any possible implementation of the second aspect.

[0139] In an eighth aspect, an embodiment of the present application provides a coding device, which includes a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call the computer program in the memory to implement the method described in the third aspect, or any possible implementation manner of the third aspect.

[0140] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium runs on a processor, it implements the method described in the first aspect, or any possible implementation of the first aspect, or the second aspect, or any possible implementation of the second aspect, or the third aspect, or any possible implementation of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0141] The following is an introduction to the drawings used in the embodiments of the present application.

[0142] Figure 1 It is a schematic diagram of the architecture of a coding and decoding system provided in an embodiment of the present application;

[0143] Figure 2 It is a flow chart of a distributed encoding and decoding method provided in an embodiment of the present application;

[0144] Figure 3 It is a schematic diagram of the relationship between input and output in image processing provided by an embodiment of the present application;

[0145] Figure 4 This is a schematic diagram of a network structure of a feedforward coding architecture provided in an embodiment of the present application;

[0146] Figure 5 It is a flow chart of a distributed encoding and decoding method provided in an embodiment of the present application;

[0147] Figure 6 is a schematic diagram of a missing pattern provided in an embodiment of the present application;

[0148] Figure 7 This is a schematic diagram of a network structure of a feedback retransmission process provided by an embodiment of the present application;

[0149] Figure 8 It is a flow chart of a distributed encoding and decoding method provided in an embodiment of the present application;

[0150] Fig. 9 is a structural diagram of a decoding device provided in an embodiment of the present application;

[0151] Fig.10 is a structural diagram of a decoding device provided in an embodiment of the present application;

[0152] Fig.11 is a schematic diagram of the structure of an encoding device provided in an embodiment of the present application;

[0153] Fig.12 is a structural schematic diagram of a decoding device provided in an embodiment of the present application;

[0154] Fig.13 It is a structural diagram of an encoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0155] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0156] See also Figure 1 , Figure 1 1 is a schematic diagram of the architecture of a coding and decoding system provided in an embodiment of the present application, the system includes a coding device 101 (also referred to as a coding end, a sending end) and a decoding device 102 (also referred to as a decoding end, a receiving end), wherein:

[0157] The encoding device 101 includes a grayscale information encoding module 1011 , which is used to encode the grayscale information of the input image x to obtain a bit stream of the grayscale information, and then send the bit stream to the decoding device 102 .

[0158] The decoding device 102 includes a grayscale information decoding module 1021, a sampling module 1022 based on optical flow, and an enhancement module 1023, wherein the grayscale information decoding module 1021 is used to receive the bit stream from the encoding device 101, and decode the grayscale information of the input image x therefrom, or it can be said that the grayscale information of the input image x is reconstructed according to the bit stream. Optionally, the grayscale information decoding module 1021 includes a receiving module and a decoding module, wherein the receiving module is used to receive the bit stream, and the decoding module is used to reconstruct the grayscale information of the input image x according to the received bit stream; the sampling module 1022 based on optical flow is used to reconstruct the grayscale information of the input image x according to the related image y and the reconstructed input image The grayscale information of x is used to determine the optical flow between the input image x and the related image y (i.e., the vector field of the position change of the pixel points corresponding to the same object between the two frames of images), and the related image y is sampled according to the optical flow to obtain a sampled image. This process uses the optical flow information to achieve pixel-level image alignment, and relatively accurately uses the side information of the related image y in the image domain. Optionally, the optical flow-based sampling module 1022 may include a determination module and a sampling module, the determination module is used to determine the optical flow, and the sampling module is used to sample to obtain a sampled image; the enhancement module 1023 is used to reconstruct the input image x according to the sampled image and the grayscale information of the reconstructed input image x. Optionally, the decoding device 102 may also include a public information extraction module 1024, the public information extraction module 1024 is used to extract the public information between the input image x and the related image y, and the public information can also be used as the input of the enhancement module 1023 like the grayscale information of the sampled image and the input image X, for reconstructing the input image x.

[0159] In an optional solution, the encoding device 101 may further include a feedback receiving module 1012 and a residual encoding module 1013, and the decoding device 102 may further include a missing pattern generation module 1025, a missing pattern feedback module 1026, a residual decoding module 1027 and a missing pattern completion module 1028, wherein on the decoding device 102, the missing pattern generation module 1025 is used to determine the part of the sampled image where there is information missing (such as RGB data missing) and the part where there is no information missing, thereby generating a missing pattern for representation, and the missing pattern feedback module 1026 is used to convert the missing pattern into a residual decoding module 1027. The feedback is fed back to the encoding device 101. Accordingly, on the encoding device 101, the feedback receiving module 1012 is used to receive the feedback information sent by the missing pattern feedback module 1026 to determine the missing part, and then the residual encoding module 1013 sends the encoding information of the part to the decoding device 102. Accordingly, the residual decoding module 1027 on the decoding device 102 receives and decodes the encoding information of the part, and the missing pattern completion module 1028 further completes the above-mentioned sampling image through the encoding information to reconstruct the above-mentioned input image x, or directly completes the previously reconstructed input image x.

[0160] See also Figure 2 , Figure 2 is a schematic diagram of a distributed encoding and decoding method provided in an embodiment of the present application. The method can be based on Figure 1 The method is implemented based on the architecture shown in the figure, and can also be implemented based on other architectures. The method includes but is not limited to the following steps:

[0161] Step S201: the encoding device sends a bit stream to the decoding device.

[0162] Among them, the bit stream includes the encoding result of the grayscale information of the input image; the input image x is the image to be encoded, and here the grayscale information of the input image x is encoded to obtain the encoding result. For example, the input image x is compressed by the encoder Enc in the encoding device to generate compressed grayscale information, and then quantized and entropy encoded to obtain a bit stream containing grayscale information. Optionally, since the quantization operation may cause gradient loss, the embodiment of the present application can add uniform noise μ in the range of [-0.5, 0.5] to the grayscale information to simulate the error caused by quantization.

[0163] Step S202: The decoding device receives a bit stream.

[0164] Step S203: The decoding device reconstructs the grayscale information of the input image according to the bit stream.

[0165] It can be understood that the grayscale information of the input image reconstructed by the decoding device may have certain losses compared with the grayscale information of the original input image of the encoding device. The information reconstruction of the input image is a mature technology and can be implemented in many ways.

[0166] Optionally, the grayscale information of the reconstructed input image corresponds to three channels.

[0167] Step S204: The decoding device determines the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image.

[0168] Specifically, the related image is an image that has an association relationship with the input image. The association relationship here refers to the existence of certain similarities in the image content of the input image and the related image. For example, the input image and the related image are images of the same scene, and there are differences between the two in shooting time or shooting angle. The target objects contained in the image content are mostly the same, for example, the same proportion exceeds a preset proportion threshold.

[0169] The optical flow information is used to characterize the vector field of the position change of each pixel point between the input image and the related image, that is, the offset value of the position of the pixel point corresponding to the same object in the input image and the position in the related image. For example, if there is an object of insect in both the input image and the related image, the offset value of the position of the pixel point corresponding to the insect in the related image compared to its position in the input image. Optionally, PWC-Net (Sun D, ​​Yang X, Liu MY, et al. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8934-8943.) based on convolutional neural network can be used as an optical flow estimator (OF-predictor) to estimate the optical flow.

[0170] Optionally, the output optical flow may be a two-channel image, where the two channels respectively represent the horizontal displacement and vertical displacement of the pixels of the same object in the two images. Of course, the optical flow may also be represented by other methods, which are not specifically limited here.

[0171] Step S205: The decoding device samples the related image through the optical flow to obtain a sampled image.

[0172] Specifically, the above optical flow information includes a vector field of position changes of many pixels in the input image and the related image. The position change vector field of some pixels is selected to compensate the position of the pixels in the related image, so that the related image can be aligned with the input image. This alignment is pixel-level alignment, and the aligned image is called a sampled image. It can be understood that under the guidance of the optical flow, the information in the area (such as pixels) corresponding to the same object of the related image and the input image is aligned.

[0173] Step S206: The decoding device reconstructs the input image according to the sampled image and the grayscale information of the reconstructed input image.

[0174] Specifically, the grayscale information of the sampled image and the input image are actually intermediate reconstruction results related to the input image. By integrating and enhancing this information, a better reconstruction result for the input image can be obtained.

[0175] In an optional solution, when the compression ratio of the encoding device is relatively high (for example, higher than a preset threshold), only a rough grayscale information can be restored when the decoding device performs decoding, which makes the optical flow calculated above inaccurate, thereby affecting the accuracy of sampling, that is, it is impossible to accurately perform pixel-level alignment and obtain an accurate sampled image. In this case, reference can be made to the method of Neural Distributed Image Compression (NDIC) to extract the common information between the sampled image and the related image, for example, the sampled image and the related image y are connected, and then sent to a common information extraction module (including an encoder-decoder structure) to obtain common information, wherein the common information includes the same information for characterizing image characteristics, and the common information extraction module can be a neural network model that can extract common information from the feature level. In this case, the above-mentioned operation of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image includes: reconstructing the input image according to the common information, the sampled image and the grayscale information of the reconstructed input image. That is to say, when the compression of the encoding by the encoding device is relatively high, the public information is required to reconstruct the input image, otherwise the public information is not required to reconstruct the input image.

[0176] Figure 3 This is a schematic diagram of the above image processing process and the relationship between related inputs and outputs. Figure 4 An optional feedforward encoding architecture network structure is provided. Figure 4 In the embodiment of the present application, the encoder on the encoding device and the decoder on the decoding device have the same structure and independent parameters. In the embodiment of the present application, the structure of the encoding structure and the decoding structure are not unique, and can be Figure 4The structure shown may also be other structures. In the training stage, the optical flow estimator used to estimate the optical flow may be any optical flow estimation model based on a neural network. Figure 4 The "splicing" in refers to the splicing operation of vectors, which can be spliced ​​in the dimension corresponding to the number of channels. Figure 4 Again, the processing of the input image from the encoding device to the decoding device is inferred: the input image x, the image dimension is (3, h, w), is decoded (or decoded) by the grayscale information decoding module to generate a grayscale information reconstruction image with a dimension of (3, h, w). Subsequently, the reconstructed grayscale information and the related image y are sent to the optical flow estimator to estimate the optical flow, the dimension of which is (2, h, w). Then, the related image y is sampled using the optical flow to obtain a sampled image with a dimension of (3, h, w). After the sampled image and the related image y are spliced, the dimension is (6, h, w), and then sent to the public information extraction module to extract the public information with a dimension of (3, h, w). Finally, the grayscale information, the sampled image and the public information are spliced, with a dimension of (9, h, w), and sent to the enhancement module to output the final reconstruction result of the input image x with a dimension of (3, h, w). It can be understood that in the above representation of image dimensions (3, h, w), 3 represents the number of channels, h represents the image height, w represents the image width, and the meanings of the remaining image dimensions are similar. It should be noted that the design of the image dimensions mentioned above is only an example.

[0177] In an optional scheme, the input image (or the sampled image) reconstructed through steps S201-S206 can be evaluated to determine whether the reconstruction result of the input image (or the sampled image) meets preset conditions. The preset conditions are used to measure the image quality or effect, such as whether the resolution is higher than the resolution reference value, whether there are any missing patterns, whether the missing amount exceeds the missing threshold, etc. If it is determined that the preset conditions are met, subsequent steps S207-S213 are executed; if it is determined that the preset conditions are not met, steps S207-S213 are not executed.

[0178] Step S207: The decoding device generates a first missing pattern corresponding to the sampled image according to the optical flow.

[0179] Specifically, if the above-mentioned reconstructed input image does not meet the preset conditions, it is generally because there is missing information in the sampling image used for reconstruction. Therefore, it is necessary to determine the first missing pattern corresponding to the sampling image. The first missing pattern marks whether there is missing information at each position divided on the sampling image. For example, it can be pre-defined that the sampling image has 64 positions (or blocks or pixels), or 128 positions (or blocks or pixels). The specific number of positions (or blocks or pixels) can be determined as needed and is not limited here.

[0180] Due to the difference in viewing angles between the input image x and the related image y, some image information can only be seen from the perspective of x and cannot be obtained by sampling y, thus forming a missing effect on the related image y. The missing pattern can be directly determined based on the optical flow generated above (actually an optical flow map, because it includes the offset of multiple pixels). The optical flow represents the vector field of the offset of the corresponding pixel points in the input image and the related image. If the offset of the pixel point exceeds the image range, then the pixel point cannot be obtained by sampling, resulting in a missing effect, such as Figure 5 The rightmost centered image 501 is shown in black on the left side, indicating that there is no image information, that is, there is information missing. The pixel points in the optical flow that exceed the image range can be marked as "1", that is, the position where there is information missing is marked with a missing value of 1, and the pixel points that do not exceed the image range are marked as "0", that is, the position where there is no information missing is marked with a missing value of 0. Figure 6 is a missing pattern, which simply illustrates the marking process. Optionally, the missing pattern can be a single-channel image composed of {0,1}. According to the missing pattern, a parameter called missing threshold is designed, which represents the maximum value of the proportion of "1" in the missing pattern, that is, the proportion of pixels that cannot be obtained by sampling. In a distributed coding scenario, the more pixels that cannot be sampled means that the less side information can be used, and the worse the reconstruction effect. In real scenarios, it is generally impossible to quantitatively judge the reconstruction results without labels, so the reconstruction results can be measured according to the missing threshold. Optionally, if the missing part of the missing pattern exceeds the missing threshold, the above-mentioned preset conditions are not met, and the feedback retransmission mechanism (i.e., steps S207-S213) is started. If the missing pattern does not exceed the missing threshold, the above-mentioned preset conditions are met, and the feedback retransmission mechanism is not started.

[0181] Step S208: The decoding device sends first indication information to the encoding device.

[0182] The first indication information is used to indicate the first missing pattern. There are multiple ways to indicate the missing pattern. For ease of understanding, an example is given below. Figure 5 It is also indicated in:

[0183] First, since the missing pattern includes missing values ​​at each position in the image, for example, 16 image blocks are divided in the image (or there are n pixels), each image block (or each pixel) corresponds to a missing value, and each image block (or each pixel) indicates that the image block (or pixel) is missing by a missing value of 0, and indicates that the image block (or pixel) is missing by a missing value of 1, then a 16-bit (or n-bit) binary number can be used to indicate which image blocks (or pixels) are missing and which image blocks (or pixels) are not missing, that is, which positions are missing and which positions are not missing. Therefore, the encoding information obtained by encoding the 16-bit (or n-bit) binary number can be used to indicate the first missing pattern. Of course, the specific number of image blocks (or pixels) into which the image is divided depends on the actual situation, so the number of binary numbers used to indicate the target position also depends on the actual situation. Of course, non-binary numbers may also be used to indicate, and the binary representation is just an example here.

[0184] The second type is that the decoding device includes a missing pattern library, the missing pattern library includes multiple missing patterns, the missing patterns are preset, for example, based on experience, or generated by a corresponding algorithm, different missing patterns in the missing pattern library have different missing positions, each missing pattern in the missing pattern library has its own corresponding number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. Figure 5 As shown, Figure 5 Four missing patterns in the missing pattern library are illustrated, and the missing area of ​​each missing pattern is different. For example, among the four missing patterns, the missing pattern in the upper left corner corresponds to the number 00, the missing pattern in the upper right corner corresponds to the number 01, the missing pattern in the lower left corner corresponds to the number 10, and the missing pattern in the lower right corner corresponds to the number 11. If the first missing pattern is most similar to the missing pattern in the lower left corner, then the first indication information includes the number 10; if the first missing pattern is most similar to the missing pattern in the upper left corner, then the first indication information includes the number 00.

[0185] Step S209: the encoding device receives the first indication information sent by the decoding device.

[0186] Step S210: the encoding device determines a first missing pattern according to first indication information.

[0187] The encoding device parses the content of the first indication information to determine the first missing pattern. For ease of understanding, an example is also given below.

[0188] If the decoding device adopts the first method mentioned above, the encoding device can determine whether the missing value of each image block in the image is 0 or 1 based on the first indication information. If the missing value is 1, it is determined that the image block is missing, and if the missing value is 0, it is determined that the image block is not missing. Therefore, the missing values ​​of all image blocks in the image are confirmed, and the first missing pattern can be restored.

[0189] If the decoding device adopts the second method, the encoding device also stores the same missing pattern library. After the encoding device parses the missing pattern number from the first indication information, it can find the corresponding missing pattern from the missing pattern library according to the number. It can be considered that the missing pattern found in the missing pattern block is the first missing pattern (the actual row is the missing pattern that is most similar to the first missing pattern). For example, if the missing pattern number parsed from the first indication information is 10, the missing pattern found by the encoding device is Figure 5 If the missing pattern number parsed from the first indication information is 00, the missing pattern found by the encoding device is Figure 5 The missing pattern in the upper left corner.

[0190] Step S211: the encoding device sends the image encoding information of the target position to the decoding device.

[0191] The target position is the position where the existence information of the first missing pattern mark is missing. It can be understood that the target position is determined on the sampled image, but the input image also has a target position. Therefore, the encoding device encodes the image information of the target position on the input image x to obtain image encoding information.

[0192] Optionally, since the encoding device has sent the grayscale information of the input image to the decoding device in the previous process, it is only necessary to encode the residual information of the complete image information at the target position relative to the grayscale information at the target position.

[0193] Step S212: The decoding device receives the image encoding information of the target position sent by the encoding device.

[0194] Step S213: The decoding device reconstructs the input image according to the image encoding information of the target position.

[0195] In an optional solution, reconstructing the input image according to the encoding information of the target position may include: reconstructing the reconstructed input image again according to the encoding information of the target position to obtain a new input image. For example, the encoding information of the target position is integrated into the input image reconstructed in the previous step S206, and the part of the target position in the reconstructed input image is enhanced to obtain a new input image.

[0196] In another alternative, if Figure 5 As shown, the reconstructing the input image according to the image coding information of the target position may include: completing the sampled image according to the image coding information of the target position, for example, splicing the residual information (i.e., the reference block) with the grayscale information (i.e., the grayscale block) of the target position, restoring the retransmitted block through a fusion module, i.e., obtaining the complete image information of the target position, and then completing the missing part in the previous sampled image through the complete image information of the target position (i.e., the restored retransmitted block), thereby obtaining the completed sampled image; thereafter, reconstructing the input image according to the completed sampled image and the grayscale information of the reconstructed input image, i.e., executing the previous step S206 again based on the completed sampled image, so that a better input image x can be reconstructed.

[0197] For ease of understanding, the following Figure 7 To illustrate with an example, Figure 7 The schematic diagram of the network structure of the feedback retransmission process is shown, in which the encoder of the encoding device and the decoder of the decoding device have the same structure and independent parameters, and of course they can also be replaced by other encoding and decoding structures. The encoding device and the decoding device have the same missing pattern library, and the missing range K of the missing pattern in the missing pattern library (referring to the K columns on the left / right side of the image) can be determined according to the offset of the two perspectives in the specific scene. The metric used by the decoding device in the process of matching the missing pattern from the missing pattern library can be the mean square error (MSE). Figure 7Taking the image processing flow as an example, the generated optical flow dimension is (2, h, w). First, the missing pattern generation module generates a first missing pattern with a dimension of (1, h, w). Then, the missing pattern feedback module selects a method for feeding back the first missing pattern. If the missing pattern feedback module directly feeds back, the missing value of the first missing pattern (for example, 1 indicates missing, and 0 indicates not missing) is sent to the encoding device after encoding and decoding. Correspondingly, the encoding device can multiply the input image with the missing value of the first missing pattern to obtain a dimension of (3, h, w). retransmitted pattern; if the missing pattern feedback module uses the missing pattern library for feedback, the first missing pattern is matched in the missing pattern library, and the number ID of the matched pattern is encoded and sent to the encoding device. The encoding device finds the matched pattern based on the number ID in the same missing pattern library and uses it as the first missing pattern. The first missing pattern obtained by the encoding device has a dimension of (1, h, w). Similarly, the input image is multiplied by the missing value of the first missing pattern to extract the retransmitted block from the input image, and the dimension is (3, h, K). After that, the retransmitted block is sent to the residual coding module to obtain the image coding information, and then the image coding information is sent to the decoding device. The residual block is restored on the decoding device and concatenated with the grayscale information transmitted in the previous feed-forward process and sent to the fusion module to restore the retransmitted block with a dimension of (3, h, w) or (3, h, K). The retransmitted block is used to complete the sampled image with a dimension of (3, h, w). The completed sampled image is then used to replace the original sampled image to reconstruct the input image.

[0198] exist Figure 2 In the described method, the optical flow is calculated based on the grayscale information of the input image and the related image, and then the input image is aligned according to the optical flow to obtain a sampled image. Since the sampled image is an aligned image, that is, aligned side information, the input image is reconstructed based on the sampled image and the grayscale information, which can more accurately and fully utilize the side information and improve the reconstruction quality.

[0199] See also Figure 8 , Figure 8 is a schematic diagram of a coding and decoding method provided in an embodiment of the present application. The method can be based on Figure 1 The method can be implemented based on the architecture shown in the figure, and can also be implemented based on other architectures. Figure 2 The process shown is summarized as follows, and the method includes but is not limited to the following steps:

[0200] Step S801: The encoding device and the decoding device interact to complete initial parameter configuration.

[0201] For example, the two parties agree that the image type is grayscale or color, the size of the transmitted image, the encoding and decoding model, the missing threshold that triggers the feedback mechanism, the feedback method of the missing pattern (such as directly encoding the missing value 1 for the missing part in the image and the missing value 0 for the non-missing part, or encoding the feedback by the number of the missing pattern in the missing pattern library), and agree to establish the same missing pattern library, etc.

[0202] Step S802: the encoding device requests transmission resources from the decoding device.

[0203] Specifically, the transmission resource is a certain number of time-frequency resource blocks, which can correspond to the number of symbols or bits that can be sent. The encoding device can determine the transmission amount according to the transmission resource, and then encode based on the transmission amount to obtain grayscale coding information.

[0204] Step S803: The encoding device sends a bit stream to the decoding device.

[0205] The method of generating the bit stream may refer to the related description of the previous step S201.

[0206] Step S804: the decoding device sends the first missing pattern (specifically, the missing value corresponding to the first missing pattern) or its corresponding number in the missing pattern library to the encoding device.

[0207] Specifically, after receiving the bit stream, the decoding device will further perform a series of operations to determine the first missing pattern and the corresponding number, for example, performing the above steps S202-S207, where step S804 is equivalent to the above step S208.

[0208] Step S805: the encoding device sends the image encoding information of the target position in the first missing pattern to the decoding device.

[0209] Specifically, after receiving the first missing pattern or the corresponding number, the encoding device will further perform a series of operations to retransmit the blocks at the positions corresponding to the first missing pattern, for example, performing the above steps S210-S211.

[0210] Step S806: the encoding device sends the grayscale information of the next image to the decoding device.

[0211] Specifically, the next image refers to the next input image. When the reconstruction of the previous input image is completed, the grayscale information of the next input image is transmitted, and the next image is reconstructed in the same way as the previous input image. The process will not be repeated here.

[0212] Step S807: the encoding device and the decoding device synchronously update the missing pattern library.

[0213] Optionally, the update may be performed regularly or periodically, or when certain trigger conditions are met, such as when a new missing pattern is generated, or when the number of new missing patterns reaches a certain number.

[0214] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0215] See also Fig. 9 , Fig. 9 It is a structural diagram of a distributed decoding device provided in an embodiment of the present application. The decoding device 90 can be a decoding device or a device or module in a decoding device. The decoding device 90 can include a receiving module 901, a decoding module 902, a determination module 903, a sampling module 904, and an enhancement module 905, wherein each unit is described in detail as follows.

[0216] A receiving module 901 is used to receive a bit stream, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0217] A decoding module 902, configured to reconstruct grayscale information of the input image according to the bit stream;

[0218] A determination module 903 is used to determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image;

[0219] A sampling module 904 is used to sample the relevant image through the optical flow to obtain a sampled image;

[0220] The enhancement module 905 is used to reconstruct the input image according to the sampled image and the grayscale information of the reconstructed input image.

[0221] In the above method, the optical flow is calculated according to the grayscale information of the input image and the related image, and then the input image is aligned according to the optical flow to obtain a sampled image. Since the sampled image is an aligned image, that is, aligned side information, the input image is reconstructed based on the sampled image and the grayscale information, which can more accurately and fully utilize the side information, thereby further improving the reconstruction quality.

[0222] In a possible implementation, the decoding device 90 further includes:

[0223] A common information extraction module, used to extract common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics;

[0224] In terms of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image, the enhancement module is specifically used for:

[0225] The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

[0226] In this way, the public information is also used as input data for reconstructing the input image, which can make up for the deficiencies of the sampled image and grayscale information and enhance the reconstruction effect of the input image.

[0227] In a possible implementation, the decoding device 90 further includes a determination module, which is used to determine that a compression ratio of an encoding device is higher than a preset threshold before extracting the common information between the sampled image and the related image.

[0228] It can be understood that when the compression ratio is higher than the preset threshold, the grayscale information has a limited effect in the reconstruction of the input image. Therefore, public information is introduced to reconstruct the input image only when the compression ratio is higher than the preset threshold, otherwise public information is not introduced. Therefore, this method can not only ensure the reconstruction effect of the input image, but also minimize the amount of calculation.

[0229] In a possible implementation, the decoding device 90 further includes:

[0230] A missing pattern generating module, used to generate a first missing pattern corresponding to the sampled image according to the optical flow, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0231] a missing pattern feedback module, configured to send first indication information to the encoding device, where the first indication information is used to indicate the first missing pattern;

[0232] a residual decoding module, configured to receive the image encoding information of the target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0233] The enhancement module 905 is further configured to reconstruct the input image according to the image coding information of the target position.

[0234] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0235] In a possible implementation, the decoding device 90 further includes an analysis module, which is used to determine whether a reconstruction result of the input image does not meet a preset condition before generating the first missing pattern corresponding to the sampled image according to the optical flow.

[0236] In this implementation, the missing part of the information is fed back to the encoding device only when the reconstruction result of the input image does not meet the preset conditions. That is, the input image is reconstructed again only when the reconstruction result does not meet the preset conditions, instead of reconstructing it again under any circumstances, thus saving computing overhead.

[0237] In a possible implementation, in the aspect of reconstructing the input image according to the image coding information of the target position, a missing pattern completion module first completes the sampled image according to the image coding information of the target position;

[0238] The enhancement module 905 is specifically configured to reconstruct the input image according to the completed sampling image and the grayscale information of the reconstructed input image.

[0239] In this method, the image encoding information at the target position is used to complete the sampled image. After the completion is completed, the previous reconstruction operation based on the sampled image and grayscale information needs to be repeated.

[0240] In a possible implementation, in terms of reconstructing the input image according to the encoding information of the target position, the enhancement module 905 is specifically configured to:

[0241] The reconstructed input image is reconstructed again according to the encoding information of the target position to obtain a new input image.

[0242] In this implementation, the encoded information of the target position can be used to directly enhance the previously reconstructed input image without completing the sampled image.

[0243] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0244] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0245] In a possible implementation manner, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0246] In a possible implementation, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0247] In a possible implementation, the device 90 further includes:

[0248] An updating module, used for updating the missing pattern position library according to the positions where information is missing in the image within a preset time period in the past;

[0249] A synchronization module is used to send the updated missing pattern library to the encoding device.

[0250] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0251] It should be noted that the implementation of each module can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0252] See also Fig.10 , Fig.10 It is a structural schematic diagram of a distributed decoding device provided in an embodiment of the present application. The decoding device can be a decoding device or a device or module in a decoding device. The decoding device can include a missing pattern generation module 1001, a missing pattern feedback module 1002, a residual decoding module 1003 and a missing pattern completion module 1004, wherein each unit is described in detail as follows.

[0253] A missing pattern generating module 1001 is used to generate a first missing pattern corresponding to a sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0254] A missing pattern feedback module 1002 is used to send first indication information to the encoding device, where the first indication information is used to indicate the first missing pattern;

[0255] The residual decoding module 1003 is used to receive the image coding information of the target position sent by the encoding device, wherein the target position is the position where the existence information of the first missing pattern mark is missing;

[0256] The missing pattern completion module 1004 is used to reconstruct the input image according to the image coding information of the target position.

[0257] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0258] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0259] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0260] In a possible implementation manner, the first indication information includes a missing value for the first missing pattern, that is, an image that needs to be sent.

[0261] In a possible implementation, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern position library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0262] In a possible implementation, the device 100 further includes:

[0263] An updating module, used for updating the missing pattern library according to the positions where information is missing in the images within a preset time period in the past;

[0264] A synchronization module is used to send the updated missing pattern library to the encoding device.

[0265] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0266] It should be noted that the implementation of each module can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0267] See also Fig.11 , Fig.11 It is a structural diagram of a distributed coding device provided in an embodiment of the present application. The coding device can be a coding device or a device or module in a coding device. The coding device 110 may include a grayscale information coding module 1101, a feedback receiving module 1102 and a residual coding module 1103, wherein each unit is described in detail as follows.

[0268] A grayscale information encoding module 1101, configured to send a bit stream to a decoding device, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0269] A feedback receiving module 1102 is configured to receive first indication information sent by a decoding device, wherein the first indication information is used to indicate a first missing pattern, and the first missing pattern marks whether there is information missing at each position divided on a sampled image, and the sampled image is an image obtained according to the bit stream and a related image of the input image;

[0270] The residual coding module 1103 is used to send the image coding information of the target position to the decoding device, wherein the target position is the position where the existence information of the first missing pattern mark is missing.

[0271] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0272] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0273] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0274] In a possible implementation manner, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0275] In a possible implementation, the method is applied to a coding device, the coding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0276] In one possible implementation:

[0277] The feedback receiving module 1102 is further configured to determine the target location according to the identifier in the first indication information;

[0278] The residual encoding module 1103 is also used to encode the image at the target position.

[0279] In a possible implementation, the encoding device 110 further includes:

[0280] The synchronization module is used to receive the updated missing pattern library sent by the decoding device.

[0281] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0282] It should be noted that the implementation of each module can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0283] See also Fig.12 , Fig.12 A decoding device 120 is provided in an embodiment of the present application. The decoding device 120 includes a processor 1201, a memory 1202 and a communication interface 1203. The processor 1201, the memory 1202 and the communication interface 1203 are interconnected via a bus.

[0284] The memory 1202 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and the memory 1202 is used for related computer programs and data. The communication interface 1203 is used to receive and send data.

[0285] The processor 1201 may be one or more central processing units (CPUs). When the processor 1201 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0286] In an optional case, the processor 1201 in the decoding device 120 is used to read the computer program code stored in the memory 1202 and perform the following operations:

[0287] Receiving a bit stream through the communication interface 1203, wherein the bit stream includes an encoding result of grayscale information of an input image;

[0288] Reconstructing grayscale information of the input image according to the bit stream;

[0289] Determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image;

[0290] Sampling the related image by the optical flow to obtain a sampled image;

[0291] The input image is reconstructed according to the sampling image and the grayscale information of the reconstructed input image.

[0292] In the above method, the optical flow is calculated according to the grayscale information of the input image and the related image, and then the input image is aligned according to the optical flow to obtain a sampled image. Since the sampled image is an aligned image, that is, aligned side information, the input image is reconstructed based on the sampled image and the grayscale information, which can more accurately and fully utilize the side information, thereby further improving the reconstruction quality.

[0293] In a possible implementation, the method further includes:

[0294] Extracting common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics;

[0295] The step of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image comprises:

[0296] The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

[0297] In this way, the public information is also used as input data for reconstructing the input image, which can make up for the deficiencies of the sampled image and grayscale information and enhance the reconstruction effect of the input image.

[0298] In a possible implementation manner, the processor 1201 is further configured to, before extracting the common information between the sampled image and the related image, determine that a compression ratio of an encoding device is higher than a preset threshold.

[0299] It can be understood that when the compression ratio is higher than the preset threshold, the grayscale information has a limited effect in the reconstruction of the input image. Therefore, public information is introduced to reconstruct the input image only when the compression ratio is higher than the preset threshold, otherwise public information is not introduced. Therefore, this method can not only ensure the reconstruction effect of the input image, but also minimize the amount of calculation.

[0300] In a possible implementation manner, the processor 1201 is further configured to:

[0301] Generating a first missing pattern corresponding to the sampled image according to the optical flow, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0302] Sending first indication information to the encoding device through the communication interface 1203, where the first indication information is used to indicate the first missing pattern;

[0303] Receiving, through the communication interface 1203, the image encoding information of the target position sent by the encoding device, wherein the target position is the position where the existence information of the first missing pattern mark is missing;

[0304] The input image is reconstructed according to the image encoding information of the target position.

[0305] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0306] In a possible implementation manner, the processor is further configured to, before generating the first missing pattern corresponding to the sampled image according to the optical flow, determine that a reconstruction result of the input image does not satisfy a preset condition.

[0307] In this implementation, the missing part of the information is fed back to the encoding device only when the reconstruction result of the input image does not meet the preset conditions. That is, the input image is reconstructed again only when the reconstruction result does not meet the preset conditions, instead of reconstructing it again under any circumstances, thus saving computing overhead.

[0308] In a possible implementation manner, in the aspect of reconstructing the input image according to the image encoding information of the target position, the processor is specifically configured to:

[0309] Completing the sampled image according to the image coding information of the target position;

[0310] The input image is reconstructed according to the completed sampling image and the grayscale information of the reconstructed input image.

[0311] In this method, the image encoding information at the target position is used to complete the sampled image. After the completion is completed, the previous reconstruction operation based on the sampled image and grayscale information needs to be repeated.

[0312] In a possible implementation manner, in the aspect of reconstructing the input image according to the encoding information of the target position, the processor is specifically configured to:

[0313] The reconstructed input image is reconstructed again according to the encoding information of the target position to obtain a new input image.

[0314] In this implementation, the encoded information of the target position can be used to directly enhance the previously reconstructed input image without completing the sampled image.

[0315] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0316] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0317] In a possible implementation manner, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0318] In a possible implementation, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0319] In a possible implementation manner, the processor 1201 is further configured to:

[0320] Update the missing pattern position library according to the positions where information is missing in the image within a preset time period in the past;

[0321] The updated missing pattern library is sent to the encoding device.

[0322] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0323] In another optional case, the processor 1201 in the decoding device 120 is used to read the computer program code stored in the memory 1202 and perform the following operations:

[0324] Generate a first missing pattern corresponding to the sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image;

[0325] Sending first indication information to the encoding device through the communication interface 1203, where the first indication information is used to indicate the first missing pattern;

[0326] Receiving, through the communication interface 1203, image coding information of a target position sent by the coding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing;

[0327] The input image is reconstructed according to the image encoding information of the target position.

[0328] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0329] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0330] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0331] In a possible implementation manner, the first indication information includes a missing value for the first missing pattern, that is, an image that needs to be sent.

[0332] In a possible implementation, the method is applied to a decoding device, the decoding device includes a missing pattern library, different missing pattern marks in the missing pattern position library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0333] In a possible implementation manner, the processor 1201 is further configured to:

[0334] Update the missing pattern library according to the positions where information is missing in the images within a preset time period in the past;

[0335] The updated missing pattern library is sent to the encoding device through the communication interface 1203 .

[0336] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0337] It should be noted that the implementation of each operation can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0338] See also Fig.13 , Fig.13 An encoding device 130 is provided in an embodiment of the present application. The encoding device 130 includes a processor 1301, a memory 1302 and a communication interface 1303. The processor 1301, the memory 1302 and the communication interface 1303 are interconnected via a bus.

[0339] The memory 1302 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), and the memory 1302 is used for related computer programs and data. The communication interface 1303 is used to receive and send data.

[0340] The processor 1301 may be one or more central processing units (CPUs). When the processor 1301 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0341] The processor 1301 in the encoding device 130 is used to read the computer program code stored in the memory 1302 and perform the following operations:

[0342] Sending a bit stream to a decoding device through the communication interface 1303, wherein the bit stream includes an encoding result of grayscale information of the input image;

[0343] Receive first indication information sent by a decoding device through a communication interface 1303, wherein the first indication information is used to indicate a first missing pattern, the first missing pattern marks whether there is information missing at each position divided on a sampled image, and the sampled image is an image obtained according to the bit stream and a related image of the input image;

[0344] The image coding information of the target position is sent to the decoding device through the communication interface 1303, wherein the target position is the position where the existence information of the first missing pattern mark is missing.

[0345] In this implementation, the decoding device feeds back the part of the sampled image where information is missing to the encoding device, and the encoding device encodes the image information of the missing part and sends it to the decoding device, so that the decoding device can reconstruct the input image based on the image information of the missing part, and obtain a higher quality input image.

[0346] In a possible implementation manner, the image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

[0347] In this implementation, only the residual information of the target position needs to be encoded, which can save transmission resources.

[0348] In a possible implementation manner, the first indication information includes the first missing pattern, that is, the missing value of the image that needs to be sent.

[0349] In a possible implementation, the method is applied to a coding device, the coding device includes a missing pattern library, different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern. It can be understood that only the number needs to be sent without sending the missing value in the missing pattern, which can save transmission resources.

[0350] In a possible implementation, the processor 1301 is further configured to, before sending the image encoding information of the target position of the first missing pattern to the decoding device, determine the target position according to an identifier in the first indication information; and encode the image at the target position.

[0351] In a possible implementation manner, the processor 1301 is further configured to:

[0352] The updated missing pattern library sent by the decoding device is received through the communication interface 1303 .

[0353] It can be understood that regularly updating the missing pattern library can ensure that the missing patterns in the missing pattern library are more similar to the missing patterns that actually appeared recently, thereby reducing the distortion of the missing patterns selected by the encoding device side based on the missing pattern library, and finally transmitting it to the reconstruction link of the input image, thereby improving the reconstruction effect of the input image.

[0354] It should be noted that the implementation of each operation can also refer to Figure 2 The corresponding description of the method embodiment shown.

[0355] The embodiment of the present application further provides a chip, the chip comprising at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor are interconnected through a line, the at least one memory stores a computer program; when the computer program is executed by the processor, Figure 2 The steps performed by the decoding device in Figure 2 The steps performed by the encoding device in.

[0356] The present application also provides a computer-readable storage medium in which a computer program is stored. When the computer-readable storage medium is executed on a processor, Figure 2 The steps performed by the decoding device in Figure 2 The steps performed by the encoding device in.

[0357] The present application also provides a computer program product, which, when executed on a processor, implements Figure 2The steps performed by the decoding device in Figure 2 The steps performed by the encoding device in.

[0358] A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program or hardware related to the computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store computer program codes.

Claims

1. A distributed decoding method, characterized in that: include: Receiving a bit stream, wherein the bit stream includes an encoding result of grayscale information of an input image; Reconstructing grayscale information of the input image according to the bit stream; Determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image; Sampling the related image by the optical flow to obtain a sampled image; The input image is reconstructed according to the sampling image and the grayscale information of the reconstructed input image.

2. The method according to claim 1, characterized in that Also includes: Extracting common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics; The step of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image comprises: The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

3. The method according to claim 2, characterized in that Before extracting the common information between the sampled image and the related image, the method further includes: It is determined that the compression ratio of the encoding device is higher than a preset threshold.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: Generating a first missing pattern corresponding to the sampled image according to the optical flow, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image; Sending first indication information to an encoding device, where the first indication information is used to indicate the first missing pattern; receiving the image encoding information of the target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing; The input image is reconstructed according to the image encoding information of the target position.

5. The method according to claim 4, characterized in that Before generating the first missing pattern corresponding to the sampled image according to the optical flow, the method further includes: It is determined that the reconstruction result of the input image does not meet a preset condition.

6. The method according to claim 4, characterized in that The step of reconstructing the input image according to the image coding information of the target position comprises: Completing the sampled image according to the image coding information of the target position; The input image is reconstructed according to the completed sampling image and the grayscale information of the reconstructed input image.

7. The method according to claim 4, characterized in that The step of reconstructing the input image according to the encoding information of the target position comprises: The reconstructed input image is reconstructed again according to the encoding information of the target position to obtain a new input image.

8. The method according to any one of claims 4 to 7, characterized in that: The image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

9. The method according to any one of claims 4 to 8, characterized in that: The first indication information includes the first missing pattern.

10. The method according to any one of claims 4 to 8, characterized in that: The method is applied to a decoding device, which includes a missing pattern library, in which different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern.

11. The method according to claim 10, characterized in that Also includes: Update the missing pattern position library according to the positions where information is missing in the image within a preset time period in the past; The updated missing pattern library is sent to the encoding device.

12. A distributed decoding method, characterized in that: include: Generate a first missing pattern corresponding to the sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image; Sending first indication information to an encoding device, where the first indication information is used to indicate the first missing pattern; receiving image encoding information of a target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing; The input image is reconstructed according to the image encoding information of the target position.

13. The method according to claim 12, characterized in that The image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

14. The method according to claim 12 or 13, characterized in that The first indication information includes the first missing pattern.

15. The method according to claim 12 or 13, characterized in that The method is applied to a decoding device, which includes a missing pattern library, where different missing pattern marks in the missing pattern position library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern.

16. The method according to claim 15, characterized in that Also includes: Update the missing pattern library according to the positions where information is missing in the images within a preset time period in the past; The updated missing pattern library is sent to the encoding device.

17. A distributed coding method, characterized in that: include: Sending a bit stream to a decoding device, wherein the bit stream includes an encoding result of grayscale information of an input image; receiving first indication information sent by a decoding device, wherein the first indication information is used to indicate a first missing pattern, the first missing pattern marks whether there is information missing at each position divided on a sampled image, the sampled image being an image obtained according to the bit stream and a related image of the input image; The image coding information of the target position is sent to the decoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing.

18. The method according to claim 17, characterized in that The image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

19. The method according to claim 17 or 18, characterized in that The first indication information includes the first missing pattern.

20. The method according to any one of claims 17 to 19, characterized in that: The method is applied to a coding device, which includes a missing pattern library, in which different missing pattern marks in the missing pattern library have different missing positions, each missing pattern in the missing pattern library corresponds to its own number, and the first indication information includes the number of the missing pattern in the missing pattern library that is closest to the first missing pattern.

21. The method according to claim 20, characterized in that Before sending the image coding information of the first missing pattern target position to the decoding device, the method further includes: Determine the target location according to the identifier in the first indication information; An image of the target location is encoded.

22. The method according to any one of claims 17 to 21, characterized in that: Also includes: The updated missing pattern library sent by the receiving decoding device is received.

23. A distributed decoding device, characterized in that: include: A receiving module, configured to receive a bit stream, wherein the bit stream includes an encoding result of grayscale information of an input image; A decoding module, used for reconstructing the grayscale information of the input image according to the bit stream; A determination module, configured to determine the optical flow between the input image and the related image according to the reconstructed grayscale information of the input image and the related image, wherein the related image is an image associated with the input image; A sampling module, used for sampling the relevant image through the optical flow to obtain a sampled image; An enhancement module is used to reconstruct the input image according to the sampled image and the grayscale information of the reconstructed input image.

24. The device according to claim 23, characterized in that Also includes: A common information extraction module, used to extract common information between the sampled image and the related image, wherein the common information includes the same information for characterizing image characteristics; In terms of reconstructing the input image according to the sampled image and the grayscale information of the reconstructed input image, the enhancement module is specifically used for: The input image is reconstructed according to the common information, the sampled image and the reconstructed grayscale information of the input image.

25. A distributed decoding device, characterized in that: include: A missing pattern generating module, used to generate a first missing pattern corresponding to the sampled image, wherein the first missing pattern marks whether there is information missing at each position divided on the sampled image; a missing pattern feedback module, configured to send first indication information to the encoding device, where the first indication information is used to indicate the first missing pattern; a residual decoding module, configured to receive image encoding information of a target position sent by the encoding device, wherein the target position is a position where the existence information of the first missing pattern mark is missing; The missing pattern completion module is used to reconstruct the input image according to the image coding information of the target position.

26. The device according to claim 25, characterized in that The image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

27. A distributed coding device, characterized in that: include: A grayscale information encoding module, used to send a bit stream to a decoding device, wherein the bit stream includes an encoding result of grayscale information of an input image; a feedback receiving module, configured to receive first indication information sent by a decoding device, wherein the first indication information is used to indicate a first missing pattern, the first missing pattern marks whether there is information missing at each position divided on a sampled image, and the sampled image is an image obtained according to the bit stream and a related image of the input image; The residual coding module is used to send the image coding information of the target position to the decoding device, wherein the target position is the position where the existence information of the first missing pattern mark is missing.

28. The device according to claim 27, characterized in that The image encoding information of the target position includes residual information of the image of the target position relative to grayscale information of the image of the target position.

29. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a processor, implements the method described in any one of claims 1 to 22.

Images