Image processing method based on implicit neural network, encoder, decoder, medium and program product

Abstract

The invention discloses an image processing method based on an implicit neural network, an encoder, a decoder, a medium and a program product, and relates to the technical field of image processing, and the method comprises the steps: initializing an encoder side implicit layer through a meta-learning method, and obtaining a meta-learning implicit layer; determining a residual hidden layer between the encoder side hidden layer and the meta learning hidden layer; respectively encoding the quantized residual hidden layer and the encoder side network to obtain encoded data, and sending the encoded data to a decoder; the method is applied to a decoder and used for receiving coded data sent by a coder end; decoding the coded data to obtain a residual hidden layer and a decoder side network; obtaining a decoder side hidden layer according to the meta learning hidden layer and the residual hidden layer; and inputting the decoder side hidden layer into the decoder side network for reasoning to obtain an original image. The coding time can be shortened, and the rate distortion performance can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, encoder, decoder, storage medium and computer program product based on implicit neural network. Background Art

[0002] The INR (Implicit Neural Representation) based codec is composed of a multi-layer perceptron network, which contains multiple entropy models and a set of hidden layers with different spatial resolutions. The encoder quantizes and entropy encodes a single image, and transmits it in the form of a binary bit stream, that is, compresses the image. The decoder entropy decodes the bit stream and performs inference to reconstruct the image. However, it takes tens of minutes or even hours for the encoder to compress an image during the encoding process. The encoding time is too long and the rate-distortion performance is poor. Summary of the invention

[0003] The main purpose of this application is to provide an image processing method, encoder, decoder, storage medium and computer program product based on implicit neural network, aiming to solve the technical problems of long encoding time and poor rate-distortion performance.

[0004] To achieve the above object, the present application proposes an image processing method based on an implicit neural network, which is applied to an encoder. The method comprises:

[0005] Initialize the encoder side hidden layer through the meta-learning method to obtain the meta-learning hidden layer;

[0006] Determine a residual hidden layer between an encoder side hidden layer and the meta-learning hidden layer;

[0007] The quantized residual hidden layer and the encoder side network are respectively encoded to obtain encoded data, and the encoded data is sent to the decoder.

[0008] In one embodiment, the step of initializing the encoder side hidden layer by a meta-learning method to obtain a meta-learning hidden layer includes:

[0009] The resolution of the meta-learning potential representation obtained by the meta-learning method is expanded, and the encoder-side hidden layer is initialized by the meta-learning potential representation after the resolution expansion to obtain the meta-learning hidden layer.

[0010] In one embodiment, the method comprises:

[0011] Obtain the distortion of the original image;

[0012] Obtaining a bit rate for residual encoding of the original image according to the encoded data;

[0013] A rate-distortion performance is obtained according to the bit rate and the distortion, and residual coding of the original image is optimized according to the rate-distortion performance.

[0014] In addition, to achieve the above purpose, the present application also proposes an image processing method based on implicit neural network, which is applied to a decoder, including:

[0015] Receive the encoded data sent by the encoder;

[0016] Decoding the encoded data to obtain a residual hidden layer and a decoder side network;

[0017] Obtain a decoder side hidden layer according to the meta-learning hidden layer and the residual hidden layer, wherein the meta-learning hidden layer is obtained by initializing the encoder side hidden layer by a meta-learning method;

[0018] The decoder side hidden layer is input into the decoder side network for inference to obtain the original image.

[0019] In one embodiment, the step of obtaining a decoder side hidden layer according to the meta-learning hidden layer and the residual hidden layer comprises:

[0020] The meta-learning hidden layer and the residual hidden layer are added element by element to obtain the decoder side hidden layer.

[0021] In one embodiment, the image processing method based on implicit neural network further includes:

[0022] The meta-learned hidden layer is pre-stored in the encoder as described above, and / or, the meta-learned hidden layer is pre-stored in the decoder as described above.

[0023] In addition, to achieve the above objectives, the present application also proposes an encoder, which is configured to implement the steps of the image processing method based on implicit neural network as described above.

[0024] In addition, to achieve the above objectives, the present application also proposes a decoder, which is configured to implement the steps of the image processing method based on implicit neural network as described above.

[0025] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the image processing method based on implicit neural network as described above are implemented.

[0026] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the image processing method based on implicit neural network as described above.

[0027] One or more technical solutions proposed in this application have at least the following technical effects:

[0028] Compared with the problems of long encoding time and poor rate-distortion performance of encoders based on implicit neural networks in the prior art, the present application first initializes the encoder-side hidden layer through a meta-learning method in the encoder to obtain the meta-learning hidden layer, and then determines the residual hidden layer according to the encoder-side hidden layer and the meta-learning hidden layer, and encodes the quantized residual hidden layer and the network into the encoded data respectively, and sends them to the decoder; at the decoder side, the residual hidden layer and the decoder-side network are decoded, and then the decoder-side hidden layer is obtained according to the residual hidden layer and the meta-learning hidden layer, and finally the decoder-side hidden layer is input into the decoder-side network for inference to obtain the original image.

[0029] Since the meta-learning hidden layer is obtained by pre-training the encoder through the meta-learning method before encoding, this pre-training enables the encoder to overfit to the corresponding image when processing the image, so that the important information in the image can be encoded more specifically in the subsequent encoding process, and the compression rate and image distortion can be well balanced. The decoder constructs the decoder side hidden layer through the residual hidden layer and the meta-learning hidden layer. Since the residual hidden layer carries the residual information related to the meta-learning hidden layer, it can use the information more effectively, which helps to restore the data more accurately at the decoding end, reduce the error caused by inaccurate encoding during decoding, and thus improve the rate-distortion performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A flowchart diagram of an embodiment of an image processing method based on an implicit neural network of the present application is provided;

[0033] Figure 2 A flowchart diagram of another embodiment of the image processing method based on implicit neural network of the present application is provided;

[0034] Figure 3 A brief flowchart of the image processing method based on implicit neural network provided in this application.

[0035] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0037] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0038] Digitalization is everywhere, and important decisions are based on big data analysis. The question of how to store information effectively is very important. Lossy compression sacrifices some information for a smaller file size. It is a key component to cope with the massive amount of image and video data uploaded, transmitted and downloaded from the Internet every day. The purpose of lossy compression is to trade off the quality of the file with the size of the file, which is called the rate-distortion trade-off.

[0039] Currently, there is an INR-based codec that has shown great potential as a low-complexity scheme for image compression due to its powerful expressive power. Generally speaking, an INR-based codec consists of a multi-layer perceptron network, which contains multiple entropy models and a set of hidden layers with different spatial resolutions. The above multiple entropy models and a set of hidden layers with different spatial resolutions are initially overfitted to a single image, then quantized, entropy encoded, and finally transmitted in the form of a binary bit stream. The decoder only needs to entropy decode the components in the bit stream and perform inference to reconstruct the image.

[0040] Despite the great potential of INR, INR-based compression is significantly limited by the encoding time. The encoding process usually takes tens of minutes or even hours to produce only one compressed image. Therefore, speeding up the encoding process has become an urgent need for INR-based codecs.

[0041] This application provides a solution to accelerate the encoding of image processing based on implicit neural networks by using meta-learning and residual coding methods.

[0042] Based on this, the embodiment of the present application provides an image processing method based on an implicit neural network, which is applied to an encoder, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of an image processing method based on an implicit neural network of the present application.

[0043] It should be noted that the execution subject of this embodiment may be an encoder, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a processor, etc. capable of realizing the above functions. The encoder is taken as an example to illustrate this embodiment and the following embodiments.

[0044] In this embodiment, the image processing method based on implicit neural network is applied to an encoder, and includes steps S10 to S30:

[0045] Step S10, initializing the encoder side hidden layer by a meta-learning method to obtain a meta-learning hidden layer;

[0046] It should be noted that meta-learning is a technology for learning how to learn. The meta-learning hidden layer is obtained by pre-training the encoder through the meta-learning method before encoding the encoder. Initializing the encoder side hidden layer through the meta-learning method is equivalent to updating the parameters of the encoder side hidden layer. The encoder side hidden layer is part of the encoder structure. During the image encoding process, it is responsible for converting the original image into a more compact representation and storing the valid information of the original image. The encoder side network is responsible for forward propagation of the encoder side hidden layer to obtain a compressed image, which is obtained by lossy compression of the original image. The encoder side network can obtain the distortion between the original image and the compressed image.

[0047] In the prior art, when performing meta-learning training, an image or the coordinates corresponding to an image is used. In one embodiment, the present application uses a batch of images to pre-train the encoder through the meta-learning method to update the model parameters of the encoder (an implicit neural network), and can identify the meta-learning initialization closer to the final INR in the weight space, ensuring the stability of the meta-learning process and enhancing the generalization ability of the meta-learning model parameters. Therefore, the present application is a new use and application of the meta-learning method in image compression.

[0048] In a feasible implementation manner, step S10 includes:

[0049] The resolution of the meta-learning potential representation obtained by the meta-learning method is expanded, and the encoder-side hidden layer is initialized by the meta-learning potential representation after the resolution expansion to obtain the meta-learning hidden layer.

[0050] It should be noted that the meta-learning potential representation is a representation form that is learned through the meta-learning algorithm and can capture the deep-level features of the data. For images, the higher the resolution, the clearer the image and the more details it contains. Expanding the resolution can make the meta-learning hidden layer universal for images of any resolution. Specifically, it includes: obtaining many images of different resolutions through upsampling or downsampling, and using images of different resolutions for meta-learning, so that the encoder can encode on images of various resolutions, reducing the resolution limitation problem in the encoding process.

[0051] In this embodiment, since the meta-learning hidden layer is universal for images of any resolution, it can process images of different resolutions more flexibly without the need for additional hidden layer adjustment or transmission for different resolutions, thereby improving the versatility and adaptability of image processing and maintaining good performance in a variety of image data processing scenarios.

[0052] Step S20, determining a residual hidden layer between the encoder side hidden layer and the meta-learning hidden layer;

[0053] It should be noted that the element-level residual of the encoder side hidden layer and the meta-learning hidden layer is calculated, where the element-level residual is the subtraction operation of the corresponding elements at the corresponding positions in the encoder side hidden layer and the meta-learning hidden layer (assuming that the residual is calculated by subtraction). For example, if an element value in the encoder side hidden layer is a, and the element value at the corresponding position in the meta-learning hidden layer is b, then the residual at this position is ab.

[0054] Since the hidden layer on the encoder side converts the image to be processed in real time into a compact form, it will keep changing, while the meta-learning hidden layer is obtained by training a batch of images before encoding and will not change during the encoding process; by performing residual calculation on each corresponding element, a new hidden layer is finally obtained, namely the residual hidden layer. This residual hidden layer reflects the difference between the encoder side hidden layer and the meta-learning hidden layer.

[0055] Step S30, respectively encode the quantized residual hidden layer and the encoder side network to obtain encoded data, and send the encoded data to the decoder.

[0056] It should be noted that the residual hidden layer is obtained by calculating the element-level residual of the encoder side hidden layer and the meta-learning hidden layer. The quantization operation is to discretize the data in this residual hidden layer to make it more suitable for encoding. For example, mapping continuous values ​​to a specific set of discrete values ​​reduces the representation range of the data and facilitates subsequent encoding operations. The encoded data is a bit stream. When the residual hidden layer and the encoder side network are encoded separately, two parts of the bit stream are obtained, and the two parts of the bit stream are decoded separately during subsequent decoding. The encoder side network is part of the encoder structure, and it and the encoder side hidden layer mentioned above together constitute an encoder. Exemplarily, the encoding method can be entropy coding.

[0057] In this embodiment, at the encoder, the encoder side hidden layer is first initialized by a meta-learning method to obtain a meta-learning hidden layer, and then the residual hidden layer is determined according to the encoder side hidden layer and the meta-learning hidden layer. The residual hidden layer and the encoder side network are quantized and encoded to obtain encoded data, which is sent to the decoder. Since the meta-learning hidden layer is obtained by pre-training the encoder by a meta-learning method, this pre-training enables the encoder to overfit to the corresponding image when processing the image, so that important information in the image can be encoded more specifically in the subsequent encoding process, and the compression rate and image distortion can be well balanced. In addition, the present application performs residual encoding on the original image, and residual encoding reduces the bit rate required for transmission, reduces the encoding time, and can accelerate the encoding process.

[0058] In a feasible implementation manner, the image processing method based on implicit neural network is applied to an encoder, and further includes steps A10 to A30:

[0059] Step A10, obtaining the distortion of the original image;

[0060] It should be noted that since the image obtained after image compression may be different from the original image, the difference from the original image is distortion. In one embodiment, image compression is equivalent to inputting the encoder hidden layer into the encoder side network for forward propagation.

[0061] Step A20, obtaining a bit rate for residual encoding of the original image according to the encoded data;

[0062] It should be noted that the coded data is obtained by residual coding the original image. Residual coding is a coding method that does not directly encode the original image, but is based on the idea of ​​residual learning. For example, the difference between the original image and the image obtained by transformation (the difference is the residual) is learned, and then the residual is encoded. This coding method helps to capture the effective information in the image more effectively and improve the coding efficiency.

[0063] The bit rate refers to the number of bits transmitted or processed per unit time. Since the coded data is the result of residual coding, the bit rate of residual coding is obtained by analyzing the coded data.

[0064] Step A30, obtaining rate-distortion performance according to the bit rate and distortion, and optimizing residual coding of the original image according to the rate-distortion performance.

[0065] It should be noted that rate-distortion performance refers to the relationship between the bit rate and distortion of the encoded data during the encoding process. The bit rate refers to the number of bits required to encode the data, while the distortion refers to the difference between the encoded data and the original data. In image coding, rate-distortion performance is an important indicator to measure the quality of the coding algorithm.

[0066] The rate-distortion performance can help select the optimal coding parameters, that is, the parameters of the encoder hidden layer. By adjusting the parameters, the encoding bit rate can be reduced while maintaining a certain image quality, or the encoding distortion can be reduced at a given bit rate.

[0067] In this embodiment, the residual coding of the original image is optimized according to the rate-distortion performance. The residual coding can better capture the features of complex images during the encoding process, thereby reducing the time required for encoding. While ensuring the image quality, the bit rate is reasonably controlled. At the same bit rate, the image distortion is smaller, making the encoded image closer to the features of the original image, reducing the image quality degradation caused by encoding.

[0068] In one embodiment, since the meta-learning hidden layer is predetermined and universal, it can provide a relatively stable basis for the decoding process regardless of the specific content of the input image, which helps to reduce decoding fluctuations caused by different image characteristics, making the decoding process more stable and reliable.

[0069] The existing method for obtaining hidden layer parameters is: starting from zero or random initialization, and determining the hidden layer parameters suitable for the current image through multiple iterations. The calculation is complex and the time required to determine the hidden layer parameters is long.

[0070] Based on this, in one feasible implementation, the meta-learning hidden layers are pre-stored in the encoder and decoder.

[0071] It should be noted that since the meta-learned hidden layer is obtained before encoding and is universal for images of any resolution, it can be stored at both the encoder and decoder and does not need to be transmitted.

[0072] In this embodiment, by pre-storing the meta-learning hidden layer in the encoder and decoder, there is no need to recalculate or obtain appropriate hidden layer initialization parameters, which can reduce the amount of initialization calculation; the meta-learning hidden layer is obtained and universal before encoding, and can be directly used to initialize the encoder side hidden layer, which greatly saves the calculation time for initialization each time encoding is performed, thereby speeding up the encoding process.

[0073] In addition, storing the meta-learning hidden layer at both ends avoids this additional transmission overhead and can use network resources more efficiently. Because the meta-learning hidden layer is closer to the initialization of the final INR (implicit neural representation) in the weight space, this allows the network to converge to the appropriate parameter configuration faster during the encoding process. The faster adjustment of network parameters during the encoding process means that better encoding effects can be achieved in a shorter time, further improving encoding efficiency.

[0074] Based on this, the embodiment of the present application provides an image processing method based on an implicit neural network, which is applied to a decoder. It should be noted that the execution subject of this embodiment can be a decoder, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a processor, etc. that can realize the above functions. The following takes the decoder as an example to illustrate this embodiment and the following embodiments.

[0075] Reference Figure 2 , Figure 2 This is a flow chart of another embodiment of the image processing method based on implicit neural network of the present application. In this embodiment, the image processing method based on implicit neural network is applied to the decoder, including steps D10 to D40:

[0076] Step D10, receiving the encoded data sent by the encoder end;

[0077] It should be noted that after the encoding is completed at the encoder end, the encoded data is transmitted to the decoder end, wherein the specific transmission method of the data transmission is not limited. The decoder receives the encoded data for subsequent decoding operations to restore the original data or perform inference operations, such as obtaining the original image in an image encoding and decoding system. Exemplarily, the encoded data is a bit stream.

[0078] Step D20, decoding the encoded data to obtain a residual hidden layer and a decoder side network;

[0079] It should be noted that the purpose of the decoding operation is to restore the residual hidden layer and the decoder side network from the encoded data. Specifically: the first bit stream obtained after entropy encoding of the residual hidden layer on the encoder side is the residual hidden layer decoded on the decoder side; on the encoder side, the second bit stream obtained after entropy encoding of the encoder side network is the decoder side network on the decoder side.

[0080] Step D30, obtaining a decoder side hidden layer according to the meta-learning hidden layer and the residual hidden layer, wherein the meta-learning hidden layer is obtained by initializing the encoder side hidden layer by a meta-learning method;

[0081] It should be noted that the meta-learning hidden layer is initialized by the meta-learning method and is the same as the meta-learning hidden layer used on the encoder side. The residual hidden layer is determined based on the encoder side hidden layer and the meta-learning hidden layer, and carries the difference information related to the meta-learning hidden layer.

[0082] In one embodiment, step D30 includes: adding the meta-learning hidden layer and the residual hidden layer element by element to obtain a decoder-side hidden layer.

[0083] It should be noted that the meta-learning hidden layer and the residual hidden layer are usually tensors with a certain dimension (such as a multidimensional array), and the element-by-element addition is to add the elements of the corresponding positions of the two tensors. By element-by-element addition, integration can be performed at the element level to obtain the decoder side hidden layer, which enables the decoder side hidden layer to integrate the initial guidance of the meta-learning hidden layer and the corrective information of the residual hidden layer, which helps to more accurately restore the original data or perform effective reasoning in subsequent decoder operations.

[0084] In this embodiment, the meta-learning hidden layer and the residual hidden layer are combined to obtain the decoder side hidden layer, which can combine the advantages of both. The meta-learning hidden layer provides initial guiding information, while the residual hidden layer supplements and corrects this initial information, thereby constructing a decoder side hidden layer that is more suitable for the decoder to perform reasoning on the original image. This construction method enables the decoder side hidden layer to make full use of the initial settings of the meta-learning hidden layer and the additional information contained in the residual hidden layer, thereby improving the accuracy and efficiency of the entire decoder in subsequent reasoning operations.

[0085] Step D40, input the decoder side hidden layer into the decoder side network for inference to obtain the original image.

[0086] It should be noted that inputting the decoder side hidden layer into the decoder side network means that the network performs inference operations on the input decoder side hidden layer data according to its own structure and pre-defined algorithms. Through this inference process, the decoder side network can gradually restore the information of the original image. The inference involves the calculation of multiple layers of neurons, and finally the processed result is restored to the original image.

[0087] In this embodiment, at the decoding end, the decoder side hidden layer is accurately constructed by using the residual hidden layer and the meta-learning hidden layer. Since the residual hidden layer carries the residual information related to the meta-learning hidden layer, the initial information of the meta-learning hidden layer and the correction information of the residual hidden layer are utilized, which helps to more accurately restore the information of the original image at the decoding end, reduce the errors caused by inaccurate encoding during decoding, and thus improve the rate-distortion performance.

[0088] For example, in order to help understand the implementation process of the image processing method based on implicit neural network obtained by combining this embodiment with the above embodiments, please refer to Figure 3 , Figure 3 A brief flowchart of an image processing method based on an implicit neural network is provided. Specifically:

[0089] Before encoding, the encoder side hidden layer is initialized by the meta-learning method to obtain the meta-learning hidden layer. At the encoder side, the encoder side hidden layer is input into the encoder side network to perform lossy compression on the original image. The residual hidden layer is obtained by element-by-element subtraction between the encoder side hidden layer and the meta-learning hidden layer. The residual hidden layer and the encoder side network are quantized and encoded respectively to obtain the encoded data (i.e., the first bit stream and the second bit stream). At the decoder side, the first bit stream and the second bit stream are decoded respectively to obtain the residual hidden layer and the decoder side network. The meta-learning hidden layer and the residual hidden layer are added element-by-element to obtain the decoder side hidden layer. Finally, the decoder side hidden layer is input into the decoder side network for inference to reconstruct the original image.

[0090] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the image processing method based on implicit neural network of the present application. More forms of simple transformations based on this technical concept, such as the interaction and combination of various embodiments, are all within the scope of protection of the present application.

[0091] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the image processing method based on implicit neural network in the above-mentioned embodiment.

[0092] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0093] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0095] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0096] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image processing method based on implicit neural networks, and can solve the technical problems of long encoding time and poor rate-distortion performance. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the image processing method based on implicit neural networks provided by the above-mentioned embodiments, and will not be described in detail here.

[0097] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned image processing method based on implicit neural network.

[0098] The computer program product provided by the present application can solve the technical problems of long encoding time and poor rate-distortion performance. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the image processing method based on implicit neural network provided by the above embodiment, which will not be described in detail here.

[0099] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image processing method based on implicit neural network, characterized in that: The image processing method based on implicit neural network is applied to an encoder, comprising: Initialize the encoder side hidden layer through the meta-learning method to obtain the meta-learning hidden layer; Determine a residual hidden layer between an encoder side hidden layer and the meta-learning hidden layer; The quantized residual hidden layer and the encoder side network are respectively encoded to obtain encoded data, and the encoded data is sent to the decoder.

2. The image processing method based on implicit neural network as claimed in claim 1, characterized in that: The step of initializing the encoder side hidden layer by the meta-learning method to obtain the meta-learning hidden layer includes: The resolution of the meta-learning potential representation obtained by the meta-learning method is expanded, and the encoder-side hidden layer is initialized by the meta-learning potential representation after the resolution expansion to obtain the meta-learning hidden layer.

3. The image processing method based on implicit neural network as claimed in claim 1, characterized in that: The method comprises: Obtain the distortion of the original image; Obtaining a bit rate for residual encoding of the original image according to the encoded data; A rate-distortion performance is obtained according to the bit rate and the distortion, and residual coding of the original image is optimized according to the rate-distortion performance.

4. An image processing method based on implicit neural network, characterized in that: The image processing method based on implicit neural network is applied to a decoder, comprising: Receive the encoded data sent by the encoder; Decoding the encoded data to obtain a residual hidden layer and a decoder side network; Obtain a decoder side hidden layer according to the meta-learning hidden layer and the residual hidden layer, wherein the meta-learning hidden layer is obtained by initializing the encoder side hidden layer by a meta-learning method; The decoder side hidden layer is input into the decoder side network for inference to obtain the original image.

5. The image processing method based on implicit neural network as claimed in claim 4, characterized in that: The step of obtaining a decoder side hidden layer according to the meta-learning hidden layer and the residual hidden layer comprises: The meta-learning hidden layer and the residual hidden layer are added element by element to obtain the decoder side hidden layer.

6. An image processing method based on implicit neural network, characterized in that: The image processing method based on implicit neural network also includes: The meta-learning hidden layer is pre-stored in the encoder as described in any one of claims 1 to 3, and / or, in the decoder as described in any one of claims 4 to 5.

7. An encoder, characterized in that The encoder implements the steps of the image processing method based on implicit neural network as described in any one of claims 1 to 3.

8. A decoder, characterized in that: The decoder implements the steps of the image processing method based on implicit neural network as described in any one of claims 4 to 5.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method based on an implicit neural network as described in any one of claims 1 to 3, and / or claims 4 to 5, and / or claim 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the image processing method based on an implicit neural network as described in any one of claims 1 to 3, and / or claims 4 to 5, and / or claim 6.

Images