Filtering method, filtering model training method, and related apparatus

Abstract

This application discloses a filtering method, a filtering model training method, and related apparatuses, belonging to the field of coding technology. The method includes determining K filtering model groups based on quantization parameters of a target image, determining a reconstruction block corresponding to a current coding block in the target image, determining a target filtering model from the K filtering model groups, and filtering the reconstruction block based on the target filtering model. The same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding qualities, and different filtering models within the same filtering model group are applicable to coding blocks having different contents. Therefore, for the reconstruction block corresponding to the current coding block, the target filtering model can be selected from the K filtering model groups with reference to the coding quality and content of the coding block, and then the reconstruction block is filtered based on the target filtering model to reduce coding distortion and improve filtering performance.

Description

[Technical Field]

[0001] [ [Technical Field] This application relates to the field of coding technology, and more particularly to filtering methods, filtering model training methods, and related apparatus. [Background technology]

[0002] Coding technology is widely applied in fields such as multimedia services, broadcasting, video communications, and storage. In the encoding process, an image is divided into multiple non-overlapping coding blocks, and these coding blocks are encoded sequentially. In the decoding process, the reconstructed blocks are sequentially parsed and extracted from the bitstream to determine the reconstructed image. However, in some cases, there may be problems with excessive non-smoothing or discontinuous pixels between neighboring reconstructed blocks, resulting in image distortion between the reconstructed image and the original image. Therefore, the reconstructed blocks need to be filtered. Furthermore, when the encoder encodes coding blocks in intra-predictive mode or inter-predictive mode, the encoder also needs to filter the reconstructed blocks to ensure the encoding quality of subsequent coding blocks.

[0003] In related technologies, a filtering model is pre-trained for each of several quantization parameters. When filtering a reconstructed block, the encoder obtains multiple filtering models by selecting filtering models from the pre-trained filtering models that correspond to several neighboring quantization parameters of the image's quantization parameters. A target filtering model is then selected from these multiple filtering models, and the reconstructed block is filtered based on the target filtering model. Furthermore, the encoder may further encode the index of the target filtering model into a bitstream and send the bitstream to the decoder. After the decoder receives the bitstream sent by the encoder, it can parse the bitstream to determine the reconstructed block and the index of the target filtering model, and then filter the reconstructed block using the target filtering model based on its index.

[0004] While one quantization parameter corresponds to one filtering model, different filtering may be required for coding blocks with different content within the same image. Therefore, the network structure of the filtering models corresponding to each quantization parameter becomes complex in order to satisfy the filtering effect for each coding block within the same image. As a result, the filtering speed in the above method is affected, and the image coding speed may be affected. [Overview of the project]

[0005] Embodiments of this application provide a filtering method, a filtering model training method, and related apparatus for improving filtering performance based on simplifying the network model and satisfying the filtering effect of coding blocks having different qualities and different content within the same image. The technical solution is as follows:

[0006] According to a first embodiment, a filtering method is provided which is applied to the encoder side. In this method, K filtering model groups are determined based on the quantization parameters of the target image, each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1, a reconstructed block corresponding to the currently coded block in the target image is determined, a target filtering model is determined from the K filtering model groups, the target filtering model is a filtering model corresponding to the smallest coding distortion that exists after the reconstructed block is filtered, the coding distortion that exists after the reconstructed block is filtered based on the target filtering model is smaller than the coding distortion of the reconstructed block, and the reconstructed block is filtered based on the target filtering model.

[0007] Optionally, the encoder obtains K reference quantization parameters from the target correspondence based on the quantization parameters of the target image. Since one quantization parameter corresponds to one filtering model group, the encoder can determine K filtering model groups based on the K reference quantization parameters.

[0008] The target correspondence indicates the correspondence between image quantization parameters and reference quantization parameters. For example, the target correspondence may be the correspondence between the quantization parameter range and the reference quantization parameter, or the correspondence between image quantization parameters and reference quantization parameters.

[0009] When the target correspondence is between a quantization parameter range and a reference quantization parameter, the reference quantization parameter is the same for all quantization parameters within the same quantization parameter range. Therefore, the encoder only needs to store the quantization parameter range and does not need to store all quantization parameters sequentially. This helps save memory space on the encoder side and improves the efficiency of the encoder in determining K filtering model groups.

[0010] When the target correspondence is the correspondence between an image quantization parameter and a reference quantization parameter, one image quantization parameter corresponds to K reference quantization parameters. Since the correlation between the K reference quantization parameters and the image quantization parameter is stronger, the correlation between the quantization parameters of the target image and the K filtering model groups determined by the encoder based on the target correspondence is also stronger. This can further improve the filtering effect.

[0011] The coding quality of a coding block is determined based on the quantization parameters corresponding to that coding block; that is, smaller quantization parameters indicate higher coding quality, and larger quantization parameters indicate lower coding quality. Furthermore, the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. Therefore, multiple coding blocks encoded based on the same quantization parameters have the same coding quality, and multiple coding blocks with the same coding quality can be filtered based on the same filtering model group. Multiple coding blocks encoded based on different quantization parameters have different coding quality, and multiple coding blocks with different coding quality can be filtered based on different filtering model groups. That is, the same filtering model group is applicable to coding blocks with the same coding quality, and different filtering model groups are applicable to coding blocks with different coding quality.

[0012] After K filtering model groups are optionally determined based on the quantization parameters of the target image, the encoder further needs to encode the quantization parameters corresponding to the K filtering model groups into a bitstream.

[0013] Optionally, the encoder determines filtering instruction information corresponding to the reconfiguration block based on the current coding block, the reconfiguration block, and K filtering model groups. This filtering instruction information indicates whether or not the reconfiguration block needs to be filtered. When the filtering instruction information indicates that the reconfiguration block needs to be filtered, the target filtering model is determined from the K filtering model groups.

[0014] The encoder inputs the reconstructed block into each of the K filtering model groups to obtain K*M filter blocks. Based on the current coding block, the reconstructed block, and the K*M filter blocks, it determines the rate distortion cost corresponding to the reconstructed block and the rate distortion cost corresponding to each filter block. If the rate distortion cost corresponding to the reconstructed block is not less than the rate distortion cost corresponding to each filter block, it determines that the filtering instruction is the first instruction, indicating that the reconstructed block needs to be filtered. Alternatively, if the rate distortion cost corresponding to the reconstructed block is less than the rate distortion cost corresponding to each filter block, it determines that the filtering instruction is the second instruction, indicating that the reconstructed block does not need to be filtered.

[0015] The rate distortion cost indicates the degree of image distortion between the reconstructed block and the original coding block, and the degree of image distortion between the filter block and the original coding block. If the rate distortion cost corresponding to a reconstructed block is smaller than the rate distortion cost corresponding to each filter block, this indicates that the image distortion between the reconstructed block and the original coding block is minimal. Thus, the image distortion between the original image and the reconstructed image restored based on the reconstructed block is minimal. In this case, the reconstructed block does not need to be filtered. If the rate distortion cost corresponding to a reconstructed block is not smaller than the rate distortion cost corresponding to each filter block, this indicates that the image distortion between the filter block and the original coding block is minimal. Thus, the image distortion between the original image and the reconstructed image restored based on the filter block is minimal. In this case, the reconstructed block needs to be filtered.

[0016] Based on the above description, an example is used in which the rate distortion cost indicates coding distortion. When filtering instruction information indicates that a reconstructed block needs to be filtered, the rate distortion costs corresponding to all filter blocks are compared with each other, and the filtering model corresponding to the filter block with the smallest rate distortion cost is determined as the target filtering model. Obviously, in practical applications, coding distortion can be further indicated by other parameters, which are not limited to the embodiments of this application.

[0017] Optionally, after determining the filtering instruction information corresponding to the reconfiguration block based on the currently coded block, the reconfiguration block, and K filtering model groups, the encoder further needs to encode the filtering instruction information into a bitstream.

[0018] After optionally selecting a target filtering model from K filtering model groups, the encoder further needs to encode the target index into a bitstream, where the target index represents the target filtering model.

[0019] It should be noted that the above is based on an example where filtering instructions indicate that a reconstructed block needs to be filtered. Clearly, in actual use, filtering instructions may alternatively indicate that a reconstructed block does not need to be filtered. When filtering instructions indicate that a reconstructed block does not need to be filtered, the reconstructed block is not filtered.

[0020] Since the same filtering model group can be applied to coding blocks having the same coding quality, different filtering model groups can be applied to coding blocks having different coding quality, and different filtering models within the same filtering model group can be applied to coding blocks having different content. In this way, after K filtering model groups are determined based on the quantization parameters of the target image, for the reconstruction block corresponding to the current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block. Then, the reconstruction block is filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks with different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks with different quality and different content within the same image can be satisfied.

[0021] According to a second embodiment, a filtering method is provided which is applied to the decoder side. In this method, K filtering model groups are determined, each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1. A reconstructed block is determined based on the bitstream, a target filtering model is determined within the K filtering model groups, and the reconstructed block is filtered based on the target filtering model.

[0022] Optionally, the decoder determines K filtering model groups based on the quantization parameters of the target image to which the reconstructed block belongs.

[0023] Optionally, after determining K filtering model groups based on the quantization parameters of the target image, the encoder further encodes the quantization parameters corresponding to the K filtering model groups into the bitstream. Therefore, after receiving the bitstream, the decoder parses and extracts the quantization parameters corresponding to the K filtering model groups from the bitstream, and can determine the K filtering model groups based on the quantization parameters corresponding to the K filtering model groups.

[0024] The decoder determines the filtering instruction information of the reconstruction block, and the filtering instruction information indicates whether the reconstruction block needs to be filtered. When the filtering instruction information indicates that the reconstruction block needs to be filtered, the target filtering model within the K filtering model groups is determined.

[0025] After determining the filtering instruction information corresponding to the reconstruction block based on the current coding block, the reconstruction block, and the K filtering model groups, the encoder further encodes the filtering instruction information into the bitstream. Therefore, after receiving the bitstream, the decoder parses and extracts the filtering instruction information from the bitstream, and can determine whether the reconstruction block needs to be filtered based on the filtering instruction information. When the filtering instruction information indicates that the reconstruction block needs to be filtered, the decoder parses and extracts the target index from the bitstream, and then can determine the target filtering model based on the target index.

[0026] Since the same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding qualities, and different filtering models within the same filtering model group are applicable to coding blocks having different contents. Thus, after K filtering model groups are determined based on the quantization parameter of the target image, for the reconstruction block corresponding to the current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and then the reconstruction block is filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks having different coding qualities and different contents within the same image, the filtering performance can be improved based on simplifying the network model, and the filtering effect of coding blocks having different qualities and different contents within the same image can be satisfied.

[0027] According to a third aspect, a filtering model training method is provided. In the method, a training sample set is obtained, the training sample set includes a plurality of sample coding blocks and reconstruction blocks corresponding to each sample coding block, and the plurality of sample coding blocks belong to One or moreThe quantization parameters of the images are the same, the filtering model to be trained is trained based on the training sample set to obtain an initial filtering model, the training sample set is divided into M initial sample subsets, each initial sample subset includes at least two sample coding blocks and at least two reconstruction blocks corresponding to the sample coding blocks, the initial filtering models are trained separately based on the M initial sample subsets to obtain M optimized filtering models, and M optimized filtering models are trained based on the training sample set to obtain one filtering model group.

[0028] Multiple sample coding blocks are obtained by splitting multiple sample images, or by splitting a single sample image. In other words, multiple sample coding blocks belong to the same group. One or more The sample coding blocks may come from the same sample image, or from different sample images, provided that the image quantization parameters are the same. Since multiple sample coding blocks are obtained by dividing an image into multiple non-overlapping coding blocks, the content of the multiple sample coding blocks will be different.

[0029] Reconstruction blocks corresponding to multiple sample coding blocks are input to an initial filtering model to obtain a filter block corresponding to each sample coding block. The peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined based on the multiple sample coding blocks and the filter block corresponding to each sample coding block. The multiple sample coding blocks are ranked in order of peak signal-to-noise ratio. The training sample set is divided into M initial sample subsets based on the ranking results. Each initial sample subset contains at least two consecutive sample coding blocks in the ranking results.

[0030] M optimization filtering models are trained using a cyclic iteration method based on the training sample set. The i-th iteration in the cyclic iteration method includes the following steps:

[0031] (1) Based on multiple sample coding blocks and reconstruction blocks corresponding to each sample coding block, the training sample set is divided into M optimization sample subsets, where the M optimization sample subsets correspond one-to-one with the M filtering models in the i-th iteration, and the M filtering models in the first iteration are the M optimization filtering models.

[0032] Reconstruction blocks corresponding to multiple sample coding blocks are input to M filtering models in the i-th iteration, obtaining M filter blocks corresponding to each sample coding block. The peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined based on the multiple sample coding blocks and the M filter blocks corresponding to each sample coding block. The training sample set is divided into M optimized sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, and each sample coding block is located in the optimized sample subset of the filtering model corresponding to the filter block with the largest peak signal-to-noise ratio among the M filter blocks corresponding to the sample coding block.

[0033] (2) Train M filtering models in the i-th iteration based on M optimized sample subsets.

[0034] (3) If i is less than the iteration threshold, the M filtering models obtained through training in the i-th iteration are used as the M filtering models in the (i+1)th iteration, and the (i+1)th iteration is executed.

[0035] (4) If i is greater than or equal to the iteration threshold, the M filtering models obtained through training in the i-th iteration are determined to be a single filtering model group.

[0036] In this application, M optimization filtering models are trained in a cyclic iterative manner. When the number of iterations i of the M filtering models is less than the iteration threshold, this indicates that the optimization filtering models obtained through the current training are unreliable. In this case, the M filtering models obtained through training in the i-th iteration are used as the M filtering models in the (i+1)-th iteration, and the (i+1)-th iteration continues. When the number of iterations i of the M filtering models is greater than or equal to the iteration threshold, this indicates that the optimization filtering models obtained through the current training are reliable. In this case, the iteration is stopped, and the M filtering models obtained through training in the i-th iteration are used as a single filtering model group.

[0037] The iteration threshold is set in advance. The iteration threshold is a specified number of iterations or the maximum number of iterations and can be set based on different requirements. This is not limited to the embodiments of this application.

[0038] In the process of splitting the training sample set into M optimized sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, it should be noted that the training sample set may be split into only one optimized sample subset. In other words, the largest peak signal-to-noise ratio among the M filter blocks corresponding to each sample coding block in the training sample set corresponds to the same filtering model. In this case, the filtering model in the i-th iteration is trained based on the one optimized sample subset obtained through the split, and iterations for other filtering models are stopped.

[0039] In this application, multiple sample coding blocks included in the training sample set belong to One or moreSince the image quantization parameters are the same, an untrained filtering model can be trained on the training sample set, and the group of filtering models obtained through training can be applied to coding blocks having the same coding quality. Furthermore, since the content of the multiple sample coding blocks included in the training sample set is different, the M filtering models included in the filtering model group can be applied to coding blocks having different content.

[0040] According to a fourth aspect, a filtering device is provided. The filtering device has functions to implement the operation of the filtering method in the first aspect. The filtering device includes at least one module. The at least one module is configured to implement the filtering method provided in the first aspect.

[0041] According to a fifth aspect, a filtering device is provided. The filtering device has functions to implement the operation of the filtering method in the second aspect. The filtering device includes at least one module. The at least one module is configured to implement the filtering method provided in the second aspect.

[0042] According to a sixth aspect, a filtering model training device is provided. The filtering device has the function of implementing the operation of the filtering model training method in the third aspect. The filtering model training device includes at least one module. The at least one module is configured to implement the filtering model training method provided in the third aspect.

[0043] According to a seventh aspect, an encoder-side device is provided. The encoder-side device includes a processor and memory, the memory being configured to store a computer program for performing a filtering method provided in a first aspect. The processor is configured to execute the computer program stored in memory to implement the filtering method in the first aspect.

[0044] Optionally, the encoder-side device may further include a communication bus. The communication bus is configured to establish a connection between the processor and memory.

[0045] According to an eighth aspect, a decoder-side device is provided. The decoder-side device includes a processor and memory, the memory being configured to store a computer program for performing a filtering method provided in a second aspect. The processor is configured to execute the computer program stored in memory to implement the filtering method in the second aspect.

[0046] Optionally, the decoder-side device may further include a communication bus. The communication bus is configured to establish a connection between the processor and memory.

[0047] According to a ninth aspect, a filtering model training device is provided. The filtering model training device includes a processor and memory, the memory being configured to store a computer program for performing a filtering model training method provided in a third aspect. The processor is configured to execute the computer program stored in memory to implement the filtering model training method in the third aspect.

[0048] Optionally, the filtering model training device may further include a communication bus. The communication bus is configured to establish a connection between the processor and memory.

[0049] According to the tenth aspect, a computer-readable storage medium is provided. The storage medium stores instructions, and when the instructions are executed on a computer, the computer is able to perform the steps of the filtering method in the first aspect, the filtering method in the second aspect, or the filtering model training method in the third aspect.

[0050] According to the eleventh aspect, a computer program product including instructions is provided. When the instructions are executed on a computer, the computer becomes capable of performing the steps of the filtering method in the first aspect, the steps of the filtering method in the second aspect, or the steps of the filtering model training method in the third aspect. In other words, a computer program is provided. When the computer program is executed on a computer, the computer becomes capable of performing the steps of the filtering method in the first aspect, the steps of the filtering method in the second aspect, or the steps of the filtering model training method in the third aspect.

[0051] The technical effects obtained in the fourth to eleventh embodiments are the same as those obtained through the corresponding technical means in the first, second, or third embodiments. Further details will not be explained here. [Brief explanation of the drawing]

[0052] [Figure 1] This is a diagram of the implementation environment according to the embodiment of this application. [Figure 2] This is an illustrative block diagram of the encoder-side structure according to an embodiment of this application. [Figure 3] This is an exemplary block diagram of the decoder-side structure according to the embodiment of this application. [Figure 4] This is a flowchart of the filtering method according to an embodiment of this application. [Figure 5] This is a flowchart of another filtering method according to an embodiment of this application. [Figure 6] This is a flowchart of the filtering model training method according to an embodiment of this application. [Figure 7] This is a diagram showing the structure of a filtering device according to an embodiment of this application. [Figure 8] This is a diagram showing the structure of another filtering device according to an embodiment of this application. [Figure 9] This is a diagram showing the structure of a filtering model training device according to an embodiment of this application. [Figure 10] This is a diagram showing the structure of a computer device according to an embodiment of this application. [Modes for carrying out the invention]

[0053] To further clarify the purpose, technical solutions, and advantages of this application, embodiments of this application will be described in more detail below with reference to the attached drawings.

[0054] Before describing in detail the filtering method provided in the embodiments of this application, we will first define the terminology and realization environment in the embodiments of this application.

[0055] To facilitate understanding, the terminology used in the embodiments of this application will be explained first.

[0056] Encoding: Encoding is the process of compressing an image into a bitstream. An image can be a still image, a moving image, or a video frame contained within a video.

[0057] Decryption: Decryption is the process of restoring an encoded bitstream to a reconstructed image according to specific syntax rules and processing methods.

[0058] Coding Block: A coding block is a coding region obtained by dividing an image to be coded. A single image may be divided into multiple coding blocks, and these multiple coding blocks collectively form an image. Each coding block may be coded independently; for example, the size of a coding block may be 128*128.

[0059] Quantization: Quantization is the process of mapping the continuous values ​​of a signal to a set of discrete amplitudes. Quantization can effectively reduce the range of values ​​in a signal, resulting in better compression, and quantization is the root cause of distortion.

[0060] Quantization parameter (QP): The quantization parameter is a crucial parameter that controls the degree of quantization and reflects the state of image compression. Generally, a smaller QP indicates finer quantization, more retained image detail, and higher encoding quality. Therefore, a higher encoding bitrate is required. A larger QP indicates coarser quantization, more severe loss of image detail, lower encoding quality, and more noticeable distortion. Therefore, a lower encoding bitrate is required. In other words, the quantization parameter is negatively correlated with the encoding bitrate.

[0061] Intra-prediction: Intra-prediction predicts the current coding block based on the reconstructed block that corresponds to the encoded coding block in the same image as the current coding block, which is located before the current coding block. For example, the current coding block is predicted based on the reconstructed block that corresponds to the encoded coding block to the left of the current coding block and the reconstructed block that corresponds to the encoded coding block above the current coding block.

[0062] Interpretation: Interpretation involves determining a reconstructed image corresponding to the encoded image located before the current image as a reference image, and predicting the current coding block based on the reconstructed blocks in the reference image that are similar to the current coding block.

[0063] The implementation environment in the embodiments of this application will be described below.

[0064] Coding technology is widely applied in fields such as multimedia services, broadcasting, video communications, and storage. In the encoding process, an image is divided into multiple non-overlapping coding blocks, and these coding blocks are encoded sequentially. In the decoding process, the reconstructed blocks are sequentially parsed and extracted from the bitstream to determine the reconstructed image. However, in some cases, there may be problems with excessive non-smoothing or discontinuous pixels between neighboring reconstructed blocks, resulting in image distortion between the reconstructed image and the original image. Therefore, the reconstructed blocks need to be filtered. Furthermore, when the encoder encodes coding blocks in intra-predictive mode or inter-predictive mode, the encoder also needs to filter the reconstructed blocks to ensure the encoding quality of subsequent coding blocks.

[0065] Figure 1 is a diagram of an implementation environment according to an embodiment of this application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 is configured to encode each coding block in an image and is further configured to filter the reconstructed blocks corresponding to the coding blocks in the process of performing encoding in intra-predictive mode or inter-predictive mode. The destination device 20 is configured to parse the bitstream to determine the reconstructed blocks and is further configured to filter the reconstructed blocks.

[0066] The source device 10 is configured to encode an image and generate a bitstream. Therefore, the source device 10 is also referred to as the image encoder or image encoder side. The destination device 20 is configured to decode the bitstream generated by the source device 10. Therefore, the destination device 20 is also referred to as the image decoder or image decoder side.

[0067] Link 30 is configured to receive the bitstream generated by the source device 10 and transmit the bitstream to the destination device 20. The storage device 40 is configured to receive the bitstream generated by the source device 10 and store the bitstream. In this case, the destination device 20 can obtain the bitstream directly from the storage device 40. Alternatively, the storage device 40 corresponds to a file server or other intermediate storage device capable of storing the bitstream generated by the source device 10. In this case, the destination device 20 can transmit the bitstream via streaming or download the bitstream stored in the storage device 40.

[0068] The source device 10 and the destination device 20 each include one or more processors and memory connected to one or more processors. The memory includes random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, and any other medium that can be used to store the necessary program code in the form of instructions or data structures and is accessible to the computer. For example, the source device 10 and the destination device 20 each include a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a handheld telephone set such as a so-called "smartphone," a television set, a camera, a display device, a digital media player, a video game console, or an in-vehicle computer.

[0069] Link 30 includes one or more media or devices capable of transmitting a bitstream from source device 10 to destination device 20. In a possible implementation, Link 30 includes one or more communication media that enable source device 10 to directly transmit a bitstream to destination device 20 in real time. In this embodiment of the application, source device 10 modulates the bitstream according to a communication standard, the communication standard being a wireless communication protocol, etc., and transmits the bitstream to destination device 20. One or more communication media include wireless communication media and / or wired communication media. For example, one or more communication media include a radio frequency (RF) spectrum or one or more physical transmission lines. One or more communication media may be part of a packet-based network. A packet-based network may be a local area network, a wide area network, a global network (e.g., the Internet), etc. One or more communication media may include routers, switches, base stations, other devices that facilitate communication from source device 10 to destination device 20, etc. This is not particularly limited in the embodiments of the application.

[0070] In a possible implementation, the storage device 40 is configured to store the received bitstream transmitted by the source device 10, and the destination device 20 can directly retrieve the bitstream from the storage device 40. In this case, the storage device 40 includes one of several distributed or locally accessed data storage media. For example, one of the several distributed or locally accessed data storage media may be a hard disk drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium configured to store a bitstream.

[0071] In possible implementations, the storage device 40 corresponds to a file server or other intermediate storage device capable of storing the bitstream generated by the source device 10, and the destination device 20 may transmit via streaming or download the image stored in the storage device 40. The file server is any type of server capable of storing the bitstream and transmitting the bitstream to the destination device 20. In possible implementations, the file server includes a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, a local disk drive, etc. The destination device 20 can obtain the bitstream through any standard data connection (including an internet connection). Any standard data connection includes a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL) or cable modem), or a combination of a wireless channel and a wired connection suitable for obtaining the bitstream stored in the file server. Transmission of the bitstream from the storage device 40 may be via streaming, download, or a combination thereof.

[0072] The implementation environment shown in Figure 1 is merely a possible implementation method. Furthermore, the technology in the embodiments of this application is not only applicable to the source device 10 that can encode an image and the destination device 20 that decodes the bitstream in Figure 1, but is also applicable to other devices that can encode an image and decode a bitstream. This is not particularly limited to the embodiments of this application.

[0073] In the implementation environment shown in Figure 1, the source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 includes a modulator / demodulator (modem) and / or a transmitter. The transmitter is also called a transmitter. The data source 120 includes an image capture device (e.g., a camera), an archive containing previously captured images, a feed-in interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these image sources.

[0074] The data source 120 is configured to transmit images to the encoder 100, and the encoder 100 is configured to encode the received images transmitted from the data source 120 to obtain a bitstream. The encoder transmits the bitstream to the output interface. In some embodiments, the source device 10 transmits the bitstream directly to the destination device 20 through the output interface 140. In other embodiments, the bitstream may, alternatively, be stored in the storage device 40 so that the destination device 20 can then obtain the bitstream for decoding and / or display.

[0075] In the implementation environment shown in Figure 1, the destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive a bitstream via link 30 and / or from storage device 40, and then transmit the bitstream to the decoder 200. The decoder 200 is configured to decode the received bitstream to obtain a reconstructed image. The decoder transmits the reconstructed image to the display device 220. The display device 220 may be integrated with the destination device 20 or located outside of the destination device 20. Generally, the display device 220 displays the reconstructed image. The display device 220 is one of several types of display devices. For example, the display device 220 is a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0076] Although not shown in Figure 1, in some embodiments, the encoder 100 and decoder 200 may be integrated with an audio encoder and an audio decoder, respectively, and may include a suitable multiplexer-demultiplexer (MUX-DEMUX) unit or other hardware and software for encoding both audio and video into the same data stream or separate data streams. In some embodiments, where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols such as the user datagram protocol (UDP).

[0077] The encoder 100 and decoder 200 may each be one of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology in the embodiments of this application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may execute the instructions in hardware through one or more processors to implement the technology in the embodiments of this application. Any one of the above (including hardware, software, and combinations of hardware and software) may be considered as one or more processors. Each of the encoder 100 and decoder 200 may be included in one or more encoders or decoders. Either the encoder or decoder can be integrated in the corresponding device as part of a combined encoder / decoder (codec).

[0078] In this embodiment of the application, the encoder 100 may generally be referred to as “signaling” or “transmitting” certain information to another device, for example, a decoder 200. The terms “signaling” or “transmitting” may generally refer to the transmission of syntax elements and / or other data used to decode a bitstream. Such transmission may occur in real time or near real time. Alternatively, such communication may occur after a period of time, for example, when the syntax elements in the encoded bitstream are stored in a computer-readable storage medium during encoding. The decoder may then retrieve the syntax elements at some point after they have been stored in the medium.

[0079] Figure 2 is an exemplary block diagram of the encoder side structure according to an embodiment of this application. The encoder side includes a predictor, a converter, a quantizer, an entropy encoder, an inverse quantizer, an inverse converter, a filter, and memory. The predictor is an intra predictor or an inter predictor. Specifically, for the current coding block in the target image to be encoded, the encoder side can perform an intra prediction on the current coding block by using an intra predictor, and can further perform an inter prediction on the current coding block by using an inter predictor. When performing an intra prediction on the current coding block, the encoder side obtains a first reference reconstruction block from memory, and based on the first reference reconstruction block, performs an intra prediction on the current coding block by using an intra predictor to obtain a prediction block corresponding to the current coding block, the first reference reconstruction block being a reconstruction block corresponding to an encoded coding block located in the target image and before the current coding block. Alternatively, when performing interpretation on the currently coded block, the encoder retrieves a second reference reconstructed block from memory, and then, based on the second reference reconstructed block, uses the interpreter to predict the currently coded block and obtain a predicted block corresponding to the currently coded block, where the second reference reconstructed block is a reconstructed block in the coded image that is similar to the currently coded block and is located before the target image.

[0080] After determining the predicted block corresponding to the currently coded block using an intra-predictor or inter-predictor according to the method described above, the encoder determines the difference between the currently coded block and the predicted block as the residual block. Next, the residual block is transformed using a converter to obtain the transformed residual block, and the transformed residual block is quantized using a quantizer to obtain the quantized and transformed residual block. Finally, the entropy encoder encodes the quantized and transformed residual block and the prediction instruction information into a bitstream and transmits the bitstream to the decoder, where the prediction instruction information indicates the prediction mode used when the currently coded block is predicted.

[0081] To ensure the coding quality of the next coding block in the vicinity of the currently coding block, before encoding the next coding block, the encoder must further perform inverse quantization on the quantized and transformed residual block using an inverse quantizer to obtain the transformed residual block, and then perform inverse transformation on the transformed residual block using an inverse converter to obtain the reconstructed residual block. The reconstructed residual block and the predicted block are then added together to obtain the reconstructed block corresponding to the currently coding block. After determining the reconstructed block corresponding to the currently coding block, the encoder filters the reconstructed block corresponding to the currently coding block using a filter according to the filtering method provided in the embodiments of this application to obtain the filtered block corresponding to the currently coding block, and then stores the filtered block corresponding to the currently coding block in memory to encode the next coding block.

[0082] Based on the above explanation, QP is a crucial parameter for controlling the degree of quantization. Therefore, in the process of encoding the currently coded block by the encoder, the QP corresponding to the currently coded block needs to be determined. In practical applications, the QPs corresponding to different coded blocks within the same image may be the same or different. For example, the encoder divides the target image into multiple non-overlapping coded blocks, and for any one of these coded blocks, the QP of the target image is used as the QP corresponding to that coded block. In another example, the encoder divides the target image into multiple non-overlapping coded blocks, and for any one of these coded blocks, the QP of the target image is used as a criterion for adaptively adjusting the QP of the coded block.

[0083] Multiple coding blocks may be of the same size or of different sizes. In other words, the encoder may divide the target image into coding blocks of the same size, or it may divide the target image into coding blocks of different sizes based on the content of the target image. The shape of the coding blocks may be square or other shapes. The shape of the coding blocks is not limited to the embodiments of this application.

[0084] The converter is optionally one of the following: a discrete cosine transform (DCT) device, a discrete sine transform (DST) device, or a Karhunen-Loeve transform (KLT) device.

[0085] Figure 3 is an exemplary block diagram of the decoder-side structure according to an embodiment of this application. The decoder-side includes an entropy decoder, a predictor, an inverse quantizer, an inverse converter, a memory, and a filter. The predictor is an intra-predictor or inter-predictor. Specifically, when the encoder-side performs intra-prediction for each coding block in the target image, the decoder-side also needs to determine the predicted block by using an intra-predictor. When the encoder-side performs inter-prediction for each coding block in the target image, the decoder-side also needs to determine the predicted block by using an inter-predictor.

[0086] After receiving the bitstream, the decoder decodes the received bitstream using an entropy decoder to obtain quantized and transformed residual blocks and prediction instruction information, the prediction instruction information indicating the prediction mode to be used when the currently coded block is predicted. The decoder then determines a specific predictor (intra predictor or inter predictor) to be used to perform the prediction based on the prediction instruction information. When it is determined that the prediction will be performed using an intra predictor, the decoder retrieves a first reference reconstructed block from memory and uses the intra predictor to determine the prediction block corresponding to the currently coded block. When it is determined that the prediction will be performed using an inter predictor, the decoder retrieves a second reference reconstructed block from memory and uses the inter predictor to determine the prediction block corresponding to the currently coded block. The quantized and transformed residual blocks are then processed sequentially using an inverse quantizer and inverse converter to obtain reconstructed residual blocks, and the reconstructed residual blocks and the prediction blocks are added together to obtain the reconstructed block corresponding to the currently coded block.

[0087] To avoid image distortion between the currently coded block and the corresponding reconstructed block, and to avoid excessive non-smoothness or discontinuous pixel issues between neighboring reconstructed blocks, the decoder can further filter the reconstructed blocks by using a filter according to the filtering method provided in the embodiments of this application.

[0088] It should be noted that the service scenarios described in embodiments of this application are intended to more clearly illustrate the technical solutions in embodiments of this application and do not constitute limitations on the technical solutions provided in embodiments of this application. Those skilled in the art will recognize that the technical solutions provided in embodiments of this application are also applicable to similar technical problems arising from the emergence of new service scenarios.

[0089] The filtering method provided in the embodiments of this application will be described in detail below.

[0090] Figure 4 is a flowchart of a filtering method according to an embodiment of this application. The method is applied to the encoder side. As shown in Figure 4, the method includes the following steps.

[0091] Step 401: Determine K filtering model groups based on the quantization parameters of the target image, each of the K filtering model groups containing M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1.

[0092] In some embodiments, the encoder obtains K reference quantization parameters from a target correspondence based on the quantization parameters of the target image. Since one quantization parameter corresponds to one filtering model group, the encoder can determine K filtering model groups based on the K reference quantization parameters.

[0093] The target correspondence indicates the correspondence between image quantization parameters and reference quantization parameters. For example, the target correspondence may be the correspondence between the quantization parameter range and the reference quantization parameter, or the correspondence between image quantization parameters and reference quantization parameters.

[0094] When the target correspondence is a correspondence between a quantization parameter range and a reference quantization parameter, the encoder first determines the quantization parameter range that includes the quantization parameter of the target image, obtains the target quantization parameter range, and then obtains K reference quantization parameters corresponding to the target quantization parameter range from the target correspondence based on the target quantization parameter range.

[0095] For example, the target correspondences are shown in Table 1. In Table 1, one quantization parameter range corresponds to three reference quantization parameters. In Table 1, for example, each quantization parameter range corresponds to three reference quantization parameters, meaning that the number of reference quantization parameters corresponding to all quantization parameter ranges is the same. Clearly, in practical applications, the number of reference quantization parameters corresponding to all quantization parameter ranges may differ as an alternative. [Table 1]

[0096] When the target correspondence is a correspondence between image quantization parameters and reference quantization parameters, the encoder directly obtains K reference quantization parameters corresponding to the quantization parameters of the target image from the target correspondence, based on the quantization parameters of the target image.

[0097] For example, the target correspondences are shown in Table 2. In Table 2, one image quantization parameter corresponds to three reference quantization parameters. In Table 2, for example, each image quantization parameter corresponds to three reference quantization parameters, meaning that the number of reference quantization parameters corresponding to all image quantization parameters is the same. Clearly, in practical applications, the number of reference quantization parameters corresponding to all image quantization parameters may differ as an alternative. [Table 2]

[0098] When the target correspondence is between a quantization parameter range and a reference quantization parameter, the reference quantization parameter is the same for all quantization parameters within the same quantization parameter range. Therefore, the encoder only needs to store the quantization parameter range and does not need to store all quantization parameters sequentially. This helps save memory space on the encoder side and improves the efficiency of the encoder in determining K filtering model groups.

[0099] When the target correspondence is the correspondence between an image quantization parameter and a reference quantization parameter, one image quantization parameter corresponds to K reference quantization parameters. Since the correlation between the K reference quantization parameters and the image quantization parameter is stronger, the correlation between the quantization parameters of the target image and the K filtering model groups determined by the encoder based on the target correspondence is also stronger. This can further improve the filtering effect.

[0100] The coding quality of a coding block is determined based on the quantization parameters corresponding to that coding block; that is, smaller quantization parameters indicate higher coding quality, and larger quantization parameters indicate lower coding quality. Furthermore, the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. Therefore, multiple coding blocks encoded based on the same quantization parameters have the same coding quality, and multiple coding blocks with the same coding quality can be filtered based on the same filtering model group. Multiple coding blocks encoded based on different quantization parameters have different coding quality, and multiple coding blocks with different coding quality can be filtered based on different filtering model groups. That is, the same filtering model group is applicable to coding blocks with the same coding quality, and different filtering model groups are applicable to coding blocks with different coding quality.

[0101] In some embodiments, after determining K filtering model groups based on the quantization parameters of the target image, the encoder further needs to encode the quantization parameters corresponding to the K filtering model groups into a bitstream. Thus, after receiving the bitstream, the decoder can parse and extract the quantization parameters corresponding to the K filtering model groups from the bitstream and determine the K filtering model groups based on the quantization parameters corresponding to the K filtering model groups.

[0102] The structure of the filtering model may be a convolutional neural network (CNN) structure or another structure. The structure of the filtering model is not limited to the embodiments of this application.

[0103] Step 402: Determine the reconstruction block that corresponds to the currently coded block in the target image.

[0104] For the process by which the encoder determines the reconstruction block corresponding to the currently coded block in the target image, please refer to the relevant explanation in Figure 2. Further details will not be explained here.

[0105] Step 403: Determine a target filtering model from K filtering model groups. The target filtering model is the filtering model that corresponds to the smallest coding distortion present after the reconstructed block has been filtered, and the coding distortion present after the reconstructed block has been filtered based on the target filtering model is smaller than the coding distortion of the reconstructed block.

[0106] In some embodiments, the encoder can determine a target filtering model from K filtering model groups by performing the following steps (1) and (2).

[0107] (1) Based on the current coding block, reconfiguration block and K filtering model groups, filtering instruction information corresponding to the reconfiguration block is determined, and the filtering instruction information indicates whether or not the reconfiguration block needs to be filtered.

[0108] The encoder inputs the reconstructed block into each of the K filtering model groups to obtain K*M filter blocks. Based on the current coding block, the reconstructed block, and the K*M filter blocks, it determines the rate distortion cost corresponding to the reconstructed block and the rate distortion cost corresponding to each filter block. If the rate distortion cost corresponding to the reconstructed block is not less than the rate distortion cost corresponding to each filter block, it determines that the filtering instruction is the first instruction, indicating that the reconstructed block needs to be filtered. Alternatively, if the rate distortion cost corresponding to the reconstructed block is less than the rate distortion cost corresponding to each filter block, it determines that the filtering instruction is the second instruction, indicating that the reconstructed block does not need to be filtered.

[0109] In one example, the rate distortion cost corresponding to the reconstructed block can be determined according to equation (1). J = D + λR (1)

[0110] In equation (1), J represents the rate distortion cost, D represents the error between the pixel value of a pixel in the reconstructed block and the pixel value of a pixel in the currently coding block, λ represents the distortion parameter, which is usually a default value, and R represents the number of bits required to encode the currently coding block into a bitstream when the reconstructed block does not need to be filtered. Obviously, in practical applications, the rate distortion cost corresponding to each filter block may also be determined according to equation (1). In this case, in equation (1), D represents the error between the pixel value of a pixel in the filter block and the pixel value of a pixel in the currently coding block, and R represents the number of bits required to encode the currently coding block into a bitstream when the reconstructed block needs to be filtered.

[0111] When the reconstructed block does not need to be filtered, the number of bits currently required to encode the coding block into a bitstream includes the number of bits required to encode the filtering instruction information, the number of bits required to encode the quantized and transformed residual block, and the number of bits required to encode the prediction instruction information. When the reconstructed block does need to be filtered, the number of bits currently required to encode the coding block into a bitstream includes the number of bits required to encode the filtering instruction information, the number of bits required to encode the quantized and transformed residual block, the number of bits required to encode the prediction instruction information, and the number of bits required to encode the filtering model index.

[0112] In some embodiments, the encoder stores a correspondence between a filtering model index and the number of bits required to encode the filtering model index. Thus, after determining K*M filtering models, the encoder can obtain the number of bits required to encode the K*M filtering model indices from the stored correspondence between the filtering model index and the number of bits required to encode the filtering model indices, based on the model indices of the K*M filtering models, and then determine the rate distortion cost corresponding to each filter block according to equation (1).

[0113] The above is based on an example where different filtering model indices require different numbers of bits to encode, i.e., different filtering model indices correspond to different numbers of bits for encoding. Clearly, in practical applications, the number of bits required to encode different filtering model indices may be the same, i.e., different filtering model indices correspond to the same number of bits for encoding. Thus, when the rate distortion cost corresponding to each filter block is determined according to equation (1), the rate distortion cost corresponding to each of the K*M filter blocks depends primarily on the pixel error between the currently coded block and the filter blocks within the K*M filter blocks.

[0114] When the rate distortion cost corresponding to a reconstructed block and the rate distortion cost corresponding to each filter block are determined according to equation (1), it should be noted that the error between the pixel value of a pixel in the reconstructed block and the pixel value of a pixel in the current coding block, and the error between the pixel value of a pixel in the current coding block and the pixel value of a pixel in each of the K*M filter blocks, are each one of the sum of absolute difference (SAD), the sum of absolute transformed difference (SATD), and the mean squared error (MSE).

[0115] The rate distortion cost indicates the degree of image distortion between the reconstructed block and the original coding block, and the degree of image distortion between the filter block and the original coding block. If the rate distortion cost corresponding to a reconstructed block is smaller than the rate distortion cost corresponding to each filter block, this indicates that the image distortion between the reconstructed block and the original coding block is minimal. Thus, the image distortion between the original image and the reconstructed image restored based on the reconstructed block is minimal. In this case, the reconstructed block does not need to be filtered. If the rate distortion cost corresponding to a reconstructed block is not smaller than the rate distortion cost corresponding to each filter block, this indicates that the image distortion between the filter block and the original coding block is minimal. Thus, the image distortion between the original image and the reconstructed image restored based on the filter block is minimal. In this case, the reconstructed block needs to be filtered.

[0116] The first and second instruction information may be in multiple forms, such as values ​​or characters. When the first and second instruction information are values, the first instruction information is 0 and the second instruction information is 1. Obviously, the values ​​of the first and second instruction information may be reversed or other values ​​as an alternative. This is not limited to the embodiments of this application.

[0117] In some embodiments, after determining the filtering instruction information corresponding to the reconfiguration block based on the currently coded block, the reconfiguration block, and K filtering model groups, the encoder further needs to encode the filtering instruction information into a bitstream. Thus, after receiving the bitstream, the decoder can determine, based on the bitstream, whether or not the reconfiguration block needs to be filtered.

[0118] (2) When the filtering instruction information indicates that the reconfiguration block needs to be filtered, the target filtering model is determined from the K filtering model groups.

[0119] Based on the above description, an example is used in which the rate distortion cost indicates coding distortion. When filtering instruction information indicates that a reconstructed block needs to be filtered, the rate distortion costs corresponding to all filter blocks are compared with each other, and the filtering model corresponding to the filter block with the smallest rate distortion cost is determined as the target filtering model. Obviously, in practical applications, coding distortion can be further indicated by other parameters, which are not limited to the embodiments of this application.

[0120] In some embodiments, after determining a target filtering model from K filtering model groups, the encoder further needs to encode a target index into the bitstream, where the target index indicates the target filtering model. Thus, after receiving the bitstream, the decoder can determine a target filtering model for filtering the reconstructed block based on the bitstream.

[0121] In one example, to distinguish between different filtering models, each filtering model corresponds to one model index, and different filtering models correspond to different model indices. In this case, the target index includes the target model index, and the target model index indicates the target filtering model within K filtering model groups.

[0122] In other examples, the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. Therefore, to distinguish between different filtering models, the same filtering model group corresponds to the same quality index, and different filtering model groups correspond to different quality indices. Different filtering models within the same filtering model group may correspond to different content indices, and filtering models within different groups may have the same content indices. In this case, the target index includes a target quality index and a target content index. The target quality index indicates the filtering model group to which the target filtering model belongs, and the target content index indicates the model within the filtering model group that is the target filtering model.

[0123] It should be noted that the above is based on an example where filtering instructions indicate that a reconstructed block needs to be filtered. Clearly, in actual use, filtering instructions may alternatively indicate that a reconstructed block does not need to be filtered. When filtering instructions indicate that a reconstructed block does not need to be filtered, the reconstructed block is not filtered.

[0124] Step 404: Filter the reconstructed blocks based on the target filtering model.

[0125] The reconstructed block is input to the target filtering model, which filters the reconstructed block by outputting a filter block according to the relevant algorithm.

[0126] In this embodiment of the application, each filtering model group comprises M filtering models, where the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. That is, the same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding quality, and different filtering models within the same filtering model group are applicable to coding blocks having different content. Thus, after K filtering model groups are determined based on the quantization parameters of the target image, for a reconstruction block corresponding to a current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and the reconstruction block is then filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks having different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks having different quality and different content within the same image can be satisfied.

[0127] Figure 5 is a flowchart of another filtering method according to an embodiment of this application. This method is applied to the decoder side. As shown in Figure 5, the method includes the following steps.

[0128] Step 501: Determine K filtering model groups, each of the K filtering model groups containing M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1.

[0129] In some embodiments, the decoder determines K filtering model groups based on the quantization parameters of the target image to which this reconstruction block belongs. For a detailed implementation process, see the relevant description in step 401. The details will not be explained again here.

[0130] In some other embodiments, after determining K filtering model groups based on the quantization parameters of the target image, the encoder further encodes the quantization parameters corresponding to the K filtering model groups into a bitstream. Thus, after receiving the bitstream, the decoder can parse and extract the quantization parameters corresponding to the K filtering model groups from the bitstream and determine the K filtering model groups based on the quantization parameters corresponding to the K filtering model groups.

[0131] Step 502: Determine the reconfiguration block based on the bitstream.

[0132] After receiving the bitstream, the decoder parses and extracts the reconstructed block corresponding to the currently coded block from the bitstream. For details on the decoder's process of parsing and extracting the reconstructed block corresponding to the currently coded block from the bitstream, please refer to the relevant explanation in Figure 3. Further details will not be explained here.

[0133] Step 503: Determine the target filtering model from the K filtering model groups.

[0134] In some embodiments, the decoder determines filtering instruction information for the reconstructed block, which indicates whether or not the reconstructed block needs to be filtered. When the filtering instruction information indicates that the reconstructed block needs to be filtered, the target filtering model within the K filtering model group is determined.

[0135] Currently, based on the coding block, the reconfiguration block, and K filtering model groups, the encoder determines the filtering instruction information corresponding to the reconfiguration block, and then further encodes the filtering instruction information into a bitstream. Therefore, after receiving the bitstream, the decoder parses and extracts the filtering instruction information from the bitstream and can determine whether or not the reconfiguration block needs to be filtered based on the filtering instruction information.

[0136] When filtering instruction information indicates that a reconfiguration block needs to be filtered, after determining the target filtering model from K filtering model groups, the encoder further encodes a target index in the bitstream to indicate the target filtering model. Thus, the decoder can further parse and extract the target index from the bitstream and then determine the target filtering model based on the target index.

[0137] Based on the above explanation, the target index may include either the target model index or the target quality index and target content index. In the cases where it does not, the process by which the decoder determines the target filtering model based on the target index is different. Therefore, the following two cases will be explained separately.

[0138] In the first case, the target index includes the target model index. In this case, the decoder directly selects the corresponding filtering model from K filtering model groups based on the target model index and determines the selected filtering model as the target filtering model.

[0139] In the second case, the target index includes a target quality index and a target content index. In this case, the decoder first selects a corresponding filtering model group from K filtering model groups based on the target quality index, and then, based on the target content index, determines the filtering model corresponding to the target content index from the selected filtering model group to obtain the target filtering model.

[0140] Step 504: Filter the reconstructed blocks based on the target filtering model.

[0141] The reconstructed block is input to the target filtering model, which filters the reconstructed block by outputting a filter block according to the relevant algorithm.

[0142] In this embodiment of the application, each filtering model group comprises M filtering models, where the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. That is, the same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding quality, and different filtering models within the same filtering model group are applicable to coding blocks having different content. Thus, after K filtering model groups are determined based on the quantization parameters of the target image, for a reconstruction block corresponding to a current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and the reconstruction block is then filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks having different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks having different quality and different content within the same image can be satisfied.

[0143] Before the encoder and decoder can filter the reconstructed block based on the relevant information in the above steps, an untrained filtering model must be trained to obtain a filtering model group corresponding to one quantization parameter. The filtering model group contains M filtering models. Figure 6 is a flowchart of a filtering model training method according to an embodiment of this application. See Figure 6. The method includes the following steps:

[0144] Step 601: Obtain a training sample set, which includes multiple sample coding blocks and a reconstruction block corresponding to each sample coding block, to which the multiple sample coding blocks belong One or more The quantization parameters of the images are the same.

[0145] Multiple sample coding blocks are obtained by splitting multiple sample images, or by splitting a single sample image. In other words, multiple sample coding blocks belong to the same group. One or more The sample coding blocks may come from the same sample image, or from different sample images, provided that the image quantization parameters are the same. Since multiple sample coding blocks are obtained by dividing an image into multiple non-overlapping coding blocks, the content of the multiple sample coding blocks will be different.

[0146] For the process of obtaining the reconstructed block corresponding to each sample coding block, please refer to the relevant explanation in Figure 2 regarding how the encoder determines the reconstructed block corresponding to the current coding block. Further details will not be explained here.

[0147] Step 602: Train the filtering model to be trained based on the training sample set to obtain the initial filtering model.

[0148] Reconstruction blocks corresponding to multiple sample coding blocks included in the training sample set are used as input to the filtering model to be trained, and multiple sample coding blocks are used as outputs to the filtering model to be trained, thereby training the filtering model and obtaining the initial filtering model.

[0149] Step 603: Divide the training sample set into M initial sample subsets, each initial sample subset containing at least two sample coding blocks and at least two reconfiguration blocks corresponding to the sample coding blocks.

[0150] Reconstruction blocks corresponding to multiple sample coding blocks are input to an initial filtering model to obtain a filter block corresponding to each sample coding block. The peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined based on the multiple sample coding blocks and the filter block corresponding to each sample coding block. The multiple sample coding blocks are ranked in order of peak signal-to-noise ratio. The training sample set is divided into M initial sample subsets based on the ranking results. Each initial sample subset contains at least two consecutive sample coding blocks in the ranking results.

[0151] For any one of the multiple sample coding blocks, the peak signal-to-noise ratio of the filter block corresponding to the sample coding block is determined according to equation (2).

number

[0152] In equation (2), PSNR represents the peak signal-to-noise ratio of the filter block corresponding to the sample coding block, n represents the number of bits required to encode each pixel in the sample coding block, which is typically 8, and MSE represents the mean squared error between the pixel value of the pixel in the sample coding block and the pixel value of the pixel in the corresponding filter block.

[0153] For example, a training sample set is evenly divided into M initial sample subsets based on the ranking results of multiple sample coding blocks, each initial sample subset containing the same number of sample coding blocks. Obviously, in practical applications, after multiple sample coding blocks have been ranked in order of peak signal-to-noise ratio, the training sample set can be further divided into M initial sample subsets according to other standards. This is not limited to the embodiments of this application.

[0154] For example, the training sample set includes 16 sample coding blocks and a reconstruction block corresponding to each sample coding block. Assume the 16 sample coding blocks are B0 to B15, and the reconstruction blocks corresponding to the 16 sample coding blocks are C0 to C15. The 16 reconstruction blocks C0 to C15 are input separately into the initial filtering model as 16 filter blocks L0 to L15. The peak signal-to-noise ratios of L0 to L15 are then determined according to equation (2), and 16 acquired peak signal-to-noise ratios PSNR0 to PSNR15 are obtained. B0 to B15 are then ranked in the order of their PSNR0 to PSNR15 values, and B0 to B15 are evenly classified into four initial sample subsets based on the ranking results of B0 to B15, each initial sample subset containing four sample coding blocks.

[0155] It should be noted that ranking multiple sample coding blocks in order of peak signal-to-noise ratio and further dividing the training sample set into M initial sample subsets is one example. In some other embodiments, the training sample set can be divided into M initial sample subsets in other ways. For example, the average pixel value corresponding to each sample coding block is determined, and the average pixel value is the average of the pixel values ​​of the pixels in the sample coding block. The multiple sample coding blocks are then ranked in order of average pixel value, and the training sample set is divided into M initial sample subsets based on the ranking result. In another example, the pixel variance corresponding to each sample coding block is determined, and the pixel variance is the variance of the pixel values ​​of the pixels in the sample coding block. The multiple sample coding blocks are then ranked in order of pixel variance, and the training sample set is divided into M initial sample subsets based on the ranking result.

[0156] Step 604: Train initial filtering models separately based on M initial sample subsets to obtain M optimized filtering models.

[0157] For any one of the M initial sample subsets, the reconstructed blocks corresponding to at least two sample coding blocks contained in the initial sample subset are used as input to the initial filtering model, and at least two sample coding blocks are used as outputs to the initial filtering model to train the initial filtering model and thereby obtain the optimized filtering model. In this way, the initial filtering model can be trained based on each of the M initial sample subsets by performing the above steps, thereby obtaining M optimized filtering models.

[0158] Based on the above explanation, we assume that the training sample set B0-B15 is equally classified into four initial sample subsets. In this case, the initial filtering models are trained separately based on the four initial sample subsets to obtain four optimized filtering models, namely filtering model A, filtering model B, filtering model C, and filtering model D.

[0159] Step 605: Train M optimized filtering models based on the training sample set to obtain one filtering model group.

[0160] M optimization filtering models are trained using a cyclic iteration method based on the training sample set. The i-th iteration in the cyclic iteration method includes the following steps:

[0161] (1) Based on multiple sample coding blocks and reconstruction blocks corresponding to each sample coding block, the training sample set is divided into M optimization sample subsets, where the M optimization sample subsets correspond one-to-one with the M filtering models in the i-th iteration, and the M filtering models in the first iteration are the M optimization filtering models.

[0162] Reconstruction blocks corresponding to multiple sample coding blocks are input to M filtering models in the i-th iteration, obtaining M filter blocks corresponding to each sample coding block. The peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined based on the multiple sample coding blocks and the M filter blocks corresponding to each sample coding block. The training sample set is divided into M optimized sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, and each sample coding block is located in the optimized sample subset of the filtering model corresponding to the filter block with the largest peak signal-to-noise ratio among the M filter blocks corresponding to the sample coding block.

[0163] For the process of determining the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, see the relevant explanation in step 603 for determining the peak signal-to-noise ratio according to equation (2). Details will not be explained again here. For any one of the multiple sample coding blocks, a filtering model is determined that corresponds to the largest peak signal-to-noise ratio among the peak signal-to-noise ratios of the M filter blocks corresponding to the sample coding block, and then the sample coding block is assigned to an optimized sample subset corresponding to the filtering model.

[0164] Based on the above description, the training sample set includes 16 sample coding blocks and reconstruction blocks corresponding to each sample coding block. Assume that the reconstruction blocks corresponding to the 16 sample coding blocks are C0 to C15. The reconstruction block C0 among the 16 reconstruction blocks is used as an example. The reconstruction block C0 is separately input into four filtering models in the i-th iteration process, and the four filter blocks corresponding to the sample coding block B0 and output by the four filtering models are L0 A 、L0 B 、L0 C and L0 D . According to formula (2), assume that the four peak signal-to-noise ratios corresponding to the sample coding block B0 are PSNR0 A 、PSNR0 B 、PSNR0 C and PSNR0 D . When the peak signal-to-noise ratio PSNR0 C is the largest, the sample coding block B0 is assigned to the optimized sample subset corresponding to the filtering model C.

[0165] (2) Based on the M optimized sample subsets, train the M filtering models in the i-th iteration process.

[0166] For any one of the M optimized sample subsets, the reconstruction block corresponding to the sample coding block included in the optimized sample subset is used as the input of the corresponding filtering model, and the sample coding block is used as the output of the corresponding filtering model to train the corresponding filtering model.

[0167] (3) If i is smaller than the iteration count threshold, use the M filtering models obtained through training in the i-th iteration process as the M filtering models in the (i + 1)-th iteration process, and execute the (i + 1)-th iteration process.

[0168] (4) If i is greater than or equal to the iteration threshold, the M filtering models obtained through training in the i-th iteration are determined to be a single filtering model group.

[0169] In this embodiment of the application, M optimization filtering models are trained in a cyclic iterative manner. When the number of iterations i of the M filtering models is less than the iteration threshold, this indicates that the optimization filtering models obtained through the current training are unreliable. In this case, the M filtering models obtained through training in the i-th iteration are used as the M filtering models in the (i+1)-th iteration, and the (i+1)-th iteration continues. When the number of iterations i of the M filtering models is greater than or equal to the iteration threshold, this indicates that the optimization filtering models obtained through the current training are reliable. In this case, the iteration is stopped, and the M filtering models obtained through training in the i-th iteration are used as a single filtering model group.

[0170] The iteration threshold is set in advance. The iteration threshold is a specified number of iterations or the maximum number of iterations and can be set based on different requirements. This is not limited to the embodiments of this application.

[0171] In the process of splitting the training sample set into M optimized sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, it should be noted that the training sample set may be split into only one optimized sample subset. In other words, the largest peak signal-to-noise ratio among the M filter blocks corresponding to each sample coding block in the training sample set corresponds to the same filtering model. In this case, the filtering model in the i-th iteration is trained based on the one optimized sample subset obtained through the split, and iterations for other filtering models are stopped.

[0172] In this embodiment of the application, a set of sample coding blocks included in the training sample set belong to One or moreSince the quantization parameters of the images are the same, an untrained filtering model is trained based on the training sample set, and one filtering model group obtained through training is applicable to coding blocks having the same coding quality. Furthermore, since the content of multiple sample coding blocks included in the training sample set is different, the M filtering models included in the filtering model group are applicable to coding blocks having different content. In this way, after K filtering model groups are determined based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and then the reconstructed block is filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks with different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks with different quality and different content within the same image can be satisfied.

[0173] Figure 7 shows the structure of a filtering device according to an embodiment of this application. The filtering device may be implemented as part of or as part of an encoder-side device by using software, hardware, or a combination thereof. The encoder-side device may be the source device shown in Figure 1. As shown in Figure 7, the device includes a first determination module 701, a second determination module 702, a third determination module 703, and a first filtering module 704.

[0174] The first decision module 701 is configured to determine K filtering model groups based on the quantization parameters of the target image, where each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0175] The second decision module 702 is configured to determine the reconstruction block corresponding to the currently coded block in the target image. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be described here.

[0176] The third decision module 703 is configured to determine a target filtering model from K filtering model groups, the target filtering model being the filtering model corresponding to the smallest coding distortion present after the reconstructed block has been filtered, and the coding distortion present after the reconstructed block has been filtered based on the target filtering model is smaller than the coding distortion of the reconstructed block. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0177] The first filtering module 704 is configured to filter the reconfigured blocks based on the target filtering model. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be explained here.

[0178] Optionally, the third decision module 703 is: Currently, there is a first decision unit configured to determine filtering instruction information corresponding to a reconfiguration block based on a coding block, a reconfiguration block, and K filtering model groups, wherein the filtering instruction information indicates whether or not the reconfiguration block needs to be filtered. When filtering instruction information indicates that a reconfiguration block needs to be filtered, a second decision unit is configured to determine a target filtering model from K filtering model groups. Includes.

[0179] Optionally, the first decision unit is: The reconstructed block is input into each of the K filtering model groups to obtain K*M filter blocks. Currently, based on the coding block, the reconstruction block, and K*M filter blocks, the rate distortion cost corresponding to the reconstruction block and the rate distortion cost corresponding to each filter block are determined. If the rate distortion cost corresponding to the reconstructed block is not less than the rate distortion cost corresponding to each filter block, then the filtering instruction information is determined to be the first instruction information, and the first instruction information indicates that the reconstructed block needs to be filtered, or If the rate distortion cost corresponding to a reconstructed block is smaller than the rate distortion cost corresponding to each filter block, it is determined that the filtering instruction is a second instruction, and the second instruction indicates that the reconstructed block does not need to be filtered. It is specifically structured in this way.

[0180] Optionally, the device is The system further includes a second filtering module configured to skip filtering a reconfiguration block when filtering instruction information indicates that the reconfiguration block does not need to be filtered.

[0181] Optionally, the third decision module 703 is: The system further includes an encoding unit configured to encode filtering instruction information into a bitstream.

[0182] Optionally, the device is A first encoding module configured to encode a target index into a bitstream, further comprising a first encoding module in which the target index represents a target filtering model.

[0183] Optionally, the device is The system further includes a second encoding module configured to encode quantization parameters corresponding to K filtering model groups into a bitstream.

[0184] In this embodiment of the application, each filtering model group comprises M filtering models, where the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. That is, the same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding quality, and different filtering models within the same filtering model group are applicable to coding blocks having different content. Thus, after K filtering model groups are determined based on the quantization parameters of the target image, for a reconstruction block corresponding to a current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and the reconstruction block is then filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks having different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks having different quality and different content within the same image can be satisfied.

[0185] It should be noted that in the filtering performed by the filtering device provided in the above embodiments, the division into functional modules described above is used merely as an illustrative example. In actual applications, the above functions may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or part of the above functions. Furthermore, the embodiments of the filtering device and filtering method provided in the above embodiments belong to the same concept. For the specific implementation process of the filtering device, refer to the embodiments of the method in detail. Details will not be described again here.

[0186] Figure 8 shows the structure of another filtering device according to an embodiment of this application. The filtering device may be implemented as part or all of a decoder-side device by using software, hardware, or a combination thereof. The decoder-side device may be the destination device shown in Figure 1. As shown in Figure 8, the device includes a first determination module 801, a second determination module 802, a third determination module 803, and a filtering module 804.

[0187] The first decision module 801 is configured to determine K filtering model groups, each of which contains M filtering models, where the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters, with both K and M being integers greater than 1. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be explained here.

[0188] The second decision module 802 is configured to determine the reconfiguration block based on the bitstream. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be explained here.

[0189] The third decision module 803 is configured to determine a target filtering model from K filtering model groups. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be explained here.

[0190] The filtering module 804 is configured to filter the reconfigured blocks based on the target filtering model. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be explained here.

[0191] Optionally, the first decision module 801 is: It is specifically configured to determine K filtering model groups based on the quantization parameters of the target image to which the reconstruction block belongs.

[0192] Optionally, the first decision module 801 is: From the bitstream, parse and extract the quantization parameters corresponding to K filtering model groups. The system is specifically configured to determine K filtering model groups based on quantization parameters corresponding to the K filtering model groups.

[0193] Optionally, the third decision module 803 is: A first decision unit configured to determine filtering instruction information for a reconfiguration block, wherein the filtering instruction information indicates whether or not the reconfiguration block needs to be filtered, and When filtering instruction information indicates that a reconfiguration block needs to be filtered, a second decision unit is configured to determine a target filtering model from K filtering model groups. Includes.

[0194] Optionally, the first decision unit is: It is specifically configured to parse and extract filtering instruction information from the bitstream.

[0195] Optionally, the third decision module 803 is: The target index is parsed and extracted from the bitstream, and the target index indicates the target filtering model. It is specifically configured to determine the target filtering model based on the target index.

[0196] In this embodiment of the application, each filtering model group comprises M filtering models, where the same filtering model group corresponds to the same quantization parameters, and different filtering model groups correspond to different quantization parameters. That is, the same filtering model group is applicable to coding blocks having the same coding quality, different filtering model groups are applicable to coding blocks having different coding quality, and different filtering models within the same filtering model group are applicable to coding blocks having different content. Thus, after K filtering model groups are determined based on the quantization parameters of the target image, for a reconstruction block corresponding to a current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and the reconstruction block is then filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks having different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks having different quality and different content within the same image can be satisfied.

[0197] It should be noted that in the filtering performed by the filtering device provided in the above embodiments, the division into functional modules described above is used merely as an illustrative example. In actual applications, the above functions may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the device is divided into different functional modules to implement all or part of the above functions. Furthermore, the embodiments of the filtering device and filtering method provided in the above embodiments belong to the same concept. For the specific implementation process of the filtering device, refer to the embodiments of the method in detail. Details will not be described again here.

[0198] Figure 9 shows the structure of a filtering model training device according to an embodiment of this application. The filtering model training device may be implemented as part of or as a whole by using software, hardware, or a combination thereof. As shown in Figure 9, the device includes an acquisition module 901, a first training module 902, a splitting module 903, a second training module 904, and a third training module 905.

[0199] The acquisition module 901 is configured to acquire a training sample set, and the training sample set includes multiple sample coding blocks and a reconstruction block corresponding to each sample coding block, to which the multiple sample coding blocks belong One or more The image quantization parameters are the same. For detailed implementation processes, please refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0200] The first training module 902 is configured to train the filtering model to be trained based on the training sample set to obtain an initial filtering model. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0201] The splitting module 903 is configured to split the training sample set into M initial sample subsets, each initial sample subset containing at least two sample coding blocks and at least two reconfiguration blocks corresponding to the sample coding blocks. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Further details will not be described again here.

[0202] The second training module 904 is configured to separately train initial filtering models based on M initial sample subsets to obtain M optimized filtering models. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0203] The third training module 905 is configured to train M optimized filtering models based on the training sample set to obtain one filtering model group. For a detailed implementation process, refer to the corresponding content in the embodiments described above. Details will not be explained again here.

[0204] Optionally, the split module 903 is: The reconstruction blocks corresponding to multiple sample coding blocks are input into the initial filtering model to obtain the filter block corresponding to each sample coding block. Based on multiple sample coding blocks and the filter blocks corresponding to each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined. Rank multiple sample coding blocks in order of peak signal-to-noise ratio, The system is specifically configured to divide the training sample set into M initial sample subsets based on the ranking results, and each sample coding block included in each initial sample subset is at least two consecutive sample coding blocks in the ranking results.

[0205] Optionally, the third training module 905 is: Based on the training sample set, the system is specifically configured to train M optimization filtering models in a cyclic iteration, where the i-th iteration in the cyclic iteration consists of the following steps: The step involves dividing the training sample set into M optimized sample subsets based on multiple sample coding blocks and a reconstruction block corresponding to each sample coding block, wherein the M optimized sample subsets correspond one-to-one with the M filtering models in the i-th iteration, and the M filtering models in the first iteration are the M optimized filtering models. The steps include: training M filtering models in the i-th iteration based on M optimized sample subsets; If i is less than the iteration threshold, the step is to use the M filtering models obtained through training in the i-th iteration as the M filtering models in the (i+1)th iteration and perform the (i+1)th iteration, or If i is greater than or equal to the iteration threshold, the M filtering models obtained through training in the i-th iteration are determined to be a single filtering model group. Includes.

[0206] Optionally, the third training module 905 is: In the i-th iteration, multiple reconstruction blocks corresponding to sample coding blocks are input into M filtering models to obtain M filter blocks corresponding to each sample coding block. Based on multiple sample coding blocks and M filter blocks corresponding to each sample coding block, the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined. The system is specifically configured to divide the training sample set into M optimization sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, where each sample coding block is located in the optimization sample subset of the filtering model corresponding to the filter block with the largest peak signal-to-noise ratio among the M filter blocks corresponding to the sample coding block.

[0207] In this embodiment of the application, a set of sample coding blocks included in the training sample set belong to One or more Since the quantization parameters of the images are the same, an untrained filtering model is trained based on the training sample set, and one filtering model group obtained through training is applicable to coding blocks having the same coding quality. Furthermore, since the content of multiple sample coding blocks included in the training sample set is different, the M filtering models included in the filtering model group are applicable to coding blocks having different content. In this way, after K filtering model groups are determined based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, the target filtering model can be selected from the K filtering model groups by referring to the coding quality and content of the coding block, and then the reconstructed block is filtered based on the target filtering model to reduce coding distortion and improve filtering performance. Furthermore, for coding blocks with different coding quality and different content within the same image, filtering performance can be improved by simplifying the network model, and the filtering effect for coding blocks with different quality and different content within the same image can be satisfied.

[0208] It should be noted that in the filtering model training performed by the filtering model training apparatus provided in the above embodiments, the division into functional modules described above is used merely as an illustrative example. In actual applications, the above functions may be assigned to different functional modules and implemented on a case-by-case basis. Specifically, the internal structure of the apparatus is divided into different functional modules to implement all or part of the above functions. Furthermore, embodiments of the filtering model training apparatus and filtering model training method provided in the above embodiments belong to the same concept. For the specific implementation process of the filtering model training apparatus, refer to the embodiments of the method in detail. Details will not be described again here.

[0209] Figure 10 is a block diagram of a computer device 1000 according to an embodiment of this application. The computer device 1000 may include a processor 1001, a memory 1002, and a bus system 1003. The processor 1001 and the memory 1002 are connected through the bus system 1003. The memory 1002 is configured to store instructions. The processor 1001 is configured to execute instructions stored in the memory 1002 to perform the filtering method and filtering model training method described in an embodiment of this application. To avoid repetition, further details are not described again here.

[0210] In this embodiment of this application, the processor 1001 may be a central processing unit (CPU), or it may be another general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0211] Memory 1002 may include a ROM device or a RAM device. Any other suitable type of storage device may also function as memory 1002. Memory 1002 may include code and data 10021 accessed by the processor 1001 via the bus 1003. Memory 1002 may further include an operating system 10023 and an application 10022. Application 10022 includes at least one program that enables the processor 1001 to perform the filtering method or filtering model training method described in embodiments of this application. For example, application 10022 may include applications 1 to N and further include applications for performing the filtering method or filtering model training method described in embodiments of this application.

[0212] Bus system 1003 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, and the like. However, for clarity, the various types of buses in the drawings are all referred to as bus system 1003.

[0213] Optionally, the computer device 1000 may further include one or more output devices, such as a display 1004. In one example, the display 1004 may be a touch-sensitive display that combines a display with a touch-sensitive unit capable of operating to sense touch input. The display 1004 may be connected to the processor 1001 via the bus 1003.

[0214] It should be noted that the computer device 1000 may perform the filtering method according to the embodiments of this application, or may perform the filtering model training method according to the embodiments of this application.

[0215] Those skilled in the art will recognize that the functions described with reference to the various exemplary logic blocks, modules, and algorithmic steps disclosed and described herein may be implemented by hardware, software, firmware, or a combination thereof. If implemented by software, the functions described with reference to the exemplary logic blocks, modules, and steps may be stored as one or more instructions or codes in a computer-readable medium or transmitted over a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or it may include any communication medium that facilitates the transmission of a computer program from one location to another (for example, according to a communication protocol). Thus, the computer-readable medium may generally correspond to (1) a non-transient tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this application. The computer program product may include a computer-readable medium.

[0216] Such computer-readable storage media may include, but are not limited to, computer-readable storage media, RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing and accessing the necessary program code in the form of instructions or data structures. Furthermore, any connection is appropriately referred to as computer-readable media. For example, if instructions are transmitted from a website, server or other remote source via coaxial cable, optical fiber, twisted pair, digital subscriber line (DSL), or by using wireless technology such as infrared, radio, or microwave, then coaxial cable, optical fiber, twisted pair, DSL, or wireless technology such as infrared, radio, or microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but in practice mean non-temporary tangible storage media. As used in this specification, disk and disc include compact discs (CDs), laser discs, optical discs, DVDs, and Blu-ray discs. A disk typically reproduces data magnetically, while a disc reproduces data optically via a laser. The combination of these should also be included within the scope of computer-readable media.

[0217] Instructions may be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or equivalent integrated circuits or discrete logic circuits. Therefore, the term “processor” as used in this specification may refer to the above-described structures or any other structures applicable to implementations of the technology described herein. Furthermore, in some embodiments, the functions described with reference to the exemplary logic blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. Furthermore, the technology may be fully implemented in one or more circuits or logic elements. For example, various exemplary logic blocks, units, and modules in encoders and decoders may be understood as corresponding circuit devices or logic elements.

[0218] The technology in embodiments of this application may be implemented in a variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in embodiments of this application to highlight the functional aspects of apparatus configured to perform the disclosed technology, but they do not necessarily have to be implemented by different hardware units. In practice, as described above, various units may be combined with appropriate software and / or firmware to form a codec hardware unit, or they may be provided by interoperable hardware units (including one or more of the above-mentioned processors).

[0219] In other words, all or part of the embodiments described above may be implemented using software, hardware, firmware, or a combination thereof. When software is used to implement the embodiments, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded into a computer and executed, all or part of the procedures or functions according to the embodiments of this application are generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted by wired (e.g., coaxial cable, optical fiber, or digital subscriber line (DSL)) or wireless (e.g., infrared, radio, or microwave) from one website, computer, server, or data center to another. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or data center that integrates one or more available media. The usable media may be magnetic media (e.g., floppy disks, hard disk drives, or magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), semiconductor media (e.g., solid-state disks (SSDs)), etc. It should be noted that the computer-readable storage media referred to in the embodiments of this application may also be non-volatile storage media, that is, non-temporary storage media.

[0220] In some embodiments, an encoder-side device is provided. The encoder-side device includes memory and a processor.

[0221] Memory is configured to store computer programs, and the processor is configured to execute the computer programs stored in memory to implement the filtering method.

[0222] In some embodiments, a decoder-side device is provided. The decoder-side device includes memory and a processor.

[0223] Memory is configured to store computer programs, and the processor is configured to execute the computer programs stored in memory to implement the filtering method.

[0224] In some embodiments, a filtering model training device is provided. The filtering model training device includes memory and a processor.

[0225] Memory is configured to store computer programs, and the processor is configured to execute the computer programs stored in memory to implement a filtering model training method.

[0226] In some embodiments, a computer-readable storage medium is provided. The storage medium stores instructions. When the instructions are executed on a computer, the computer becomes capable of performing the steps of the method described above.

[0227] In some embodiments, a computer program is provided. When the computer program is executed, the above method is realized.

[0228] It should be understood that “multiple” in this specification means two or more. In the description of embodiments of this application, unless otherwise specified, “ / ” means “or.” For example, A / B may indicate A or B. In this specification, “and / or” describes only the relationship between related objects and indicates that three relationships may exist. For example, A and / or B may indicate the following three cases: that only A exists, that both A and B exist, and that only B exists. Furthermore, in order to clearly describe the technical solutions in embodiments of this application, terms such as “first” and “second” are used in embodiments of this application to distinguish between the same or similar items that provide essentially the same function or purpose. Those skilled in the art will understand that terms such as “first” and “second” do not limit the number or order of execution, and that terms such as “first” and “second” do not indicate a clear distinction.

[0229] It should be noted that information (including, but not limited to, user device information, user personal information, etc.), data (including, but not limited to, data used for analysis, stored data, displayed data, etc.) and signals in the embodiments of this application are used with the permission of the user or with the full permission of all parties, and the capture, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the quantization parameters, filtering models, current coding blocks and reconstructed blocks in the embodiments of this application are all obtained with full permission.

[0230] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modification, equivalent substitution or improvement made without departing from the spirit and principles of this application should fall within the scope of protection of this application.

Claims

1. A filtering method applied to the encoder side, The steps include determining K filtering model groups based on K reference quantization parameters obtained from target correspondence relationships based on the quantization parameters of the target image, wherein each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1, and The steps include determining the reconstruction block corresponding to the currently coded block in the target image, A step of determining a target filtering model from the K filtering model groups, wherein the target filtering model is a filtering model corresponding to the smallest coding distortion present after the reconstructed block has been filtered, and the coding distortion present after the reconstructed block has been filtered based on the target filtering model is smaller than the coding distortion of the reconstructed block. A step of filtering the reconstructed block based on the target filtering model. A method that includes this.

2. The step of determining a target filtering model from the aforementioned K filtering model groups is: The step of determining filtering instruction information corresponding to the reconfiguration block based on the current coding block, the reconfiguration block, and the K filtering model groups, wherein the filtering instruction information indicates whether or not the reconfiguration block needs to be filtered. When the filtering instruction information indicates that the reconfiguration block needs to be filtered, the step of determining the target filtering model from the K filtering model group: The method according to claim 1, including the method described in claim 1.

3. The step of determining filtering instruction information corresponding to the reconfiguration block based on the current coding block, the reconfiguration block, and the K filtering model groups is: The steps include inputting the aforementioned reconstruction block into each of the K filtering model groups to obtain K*M filter blocks, A step of determining the rate distortion cost corresponding to the reconfiguration block and the rate distortion cost corresponding to each filter block, based on the current coding block, the reconfiguration block, and the K*M filter blocks. A step in which the filtering instruction information is determined to be first instruction information if the rate distortion cost corresponding to the reconstruction block is not less than the rate distortion cost corresponding to each filter block, and the first instruction information indicates that the reconstruction block needs to be filtered, or The step of determining that the filtering instruction information is second instruction information if the rate distortion cost corresponding to the reconstruction block is smaller than the rate distortion cost corresponding to each filter block, and the second instruction information indicates that the reconstruction block does not need to be filtered. The method according to claim 2, including the method described in claim 2.

4. The method according to claim 2, further comprising the step of skipping filtering the reconstructed block when the filtering instruction information indicates that the reconstructed block does not need to be filtered.

5. After determining the filtering instruction information corresponding to the reconfiguration block based on the current coding block, the reconfiguration block, and the K filtering model groups, The method according to claim 2, further comprising the step of encoding the filtering instruction information into a bitstream.

6. After determining the target filtering model from the aforementioned K filtering model groups, The method according to claim 1 or 2, further comprising the step of encoding a target index into a bitstream, wherein the target index represents the target filtering model.

7. After determining K filtering model groups based on the quantization parameters of the target image, The method according to claim 1 or 2, further comprising the step of encoding quantization parameters corresponding to the K filtering model groups into a bitstream.

8. A filtering method applied to the decoder side, The step of determining K filtering model groups, where each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1. A step of determining the reconfiguration block based on the bitstream, The steps include determining a target filtering model from the aforementioned K filtering model groups, A step of filtering the reconstructed block based on the target filtering model. A method that includes this.

9. The step of determining K filtering model groups is: The method according to claim 8, comprising the step of determining the K filtering model groups based on K reference quantization parameters obtained from a target correspondence relationship based on the quantization parameters of the target image to which the reconstruction block belongs.

10. The step of determining K filtering model groups is: The steps include: parsing and extracting quantization parameters corresponding to the K filtering model groups from the bitstream; A step of determining the K filtering model groups based on the quantization parameters corresponding to the K filtering model groups. The method according to claim 8, including the method described in claim 8.

11. The step of determining a target filtering model from the aforementioned K filtering model groups is: The step of determining filtering instruction information for the reconfiguration block, wherein the filtering instruction information indicates whether or not the reconfiguration block needs to be filtered. When the filtering instruction information indicates that the reconfiguration block needs to be filtered, the step of determining the target filtering model from the K filtering model group: The method according to claim 8 or 9, including the method described in claim 8 or 9.

12. The step of determining the filtering instruction information for the reconfiguration block is: The method according to claim 11, further comprising the step of parsing and extracting the filtering instruction information from the bitstream.

13. The step of determining a target filtering model from the aforementioned K filtering model groups is: The step of parsing and extracting the target index from the bitstream, wherein the target index represents the target filtering model, and the step of The step of determining the target filtering model based on the target index. The method according to claim 8 or 9, including the method described in claim 8 or 9.

14. A filtering model training method, Step 1: Obtain a training sample set, wherein the training sample set includes a plurality of sample coding blocks and a reconstruction block corresponding to each sample coding block, and the quantization parameters of one or more images to which the plurality of sample coding blocks belong are the same. The steps include: training the filtering model to be trained based on the aforementioned training sample set to obtain an initial filtering model; The step of dividing the training sample set into M initial sample subsets, each initial sample subset comprising at least two sample coding blocks and a reconfiguration block corresponding to the at least two sample coding blocks, The steps include: training the initial filtering models separately based on the M initial sample subsets to obtain M optimized filtering models; The steps include: training the M optimization filtering models based on the training sample set to obtain one filtering model group; A method that includes this.

15. The step of dividing the aforementioned training sample set into M initial sample subsets is: The steps include inputting the reconstruction blocks corresponding to the plurality of sample coding blocks into the initial filtering model to obtain a filter block corresponding to each sample coding block, A step of determining the peak signal-to-noise ratio of the filter block corresponding to each sample coding block, based on the plurality of sample coding blocks and the filter block corresponding to each sample coding block, A step of ranking the plurality of sample coding blocks in order of peak signal-to-noise ratio, The steps include dividing the training sample set into M initial sample subsets based on the ranking results, wherein each sample coding block included in the initial sample subset is at least two consecutive sample coding blocks in the ranking results, and The method according to claim 14, including the method described in claim 14.

16. The step of training the M optimization filtering models based on the aforementioned training sample set to obtain one filtering model group is: The process includes training the M optimization filtering models in a cyclic iteration manner based on the aforementioned training sample set, wherein the i-th iteration in the cyclic iteration manner is as follows: The step of dividing the training sample set into M optimization sample subsets based on the plurality of sample coding blocks and the reconstruction block corresponding to each sample coding block, wherein the M optimization sample subsets correspond one-to-one with the M filtering models in the i-th iteration, and the M filtering models in the first iteration are the M optimization filtering models. The steps include training the M filtering models in the i-th iteration based on the M optimization sample subsets, If i is less than the iteration threshold, the steps are to use the M filtering models obtained through training in the i-th iteration as the M filtering models in the (i+1)th iteration and perform the (i+1)th iteration, or If i is greater than or equal to the iteration threshold, the step of determining the M filtering models obtained through training in the i-th iteration as the one filtering model group, The method according to claim 14 or 15, including the method described in claim 14 or 15.

17. The step of dividing the training sample set into M optimization sample subsets based on the plurality of sample coding blocks and the reconstruction block corresponding to each sample coding block is: The i-th iteration process involves inputting the reconstruction blocks corresponding to the plurality of sample coding blocks into the M filtering models to obtain M filter blocks corresponding to each sample coding block, A step of determining the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, based on the plurality of sample coding blocks and the M filter blocks corresponding to each sample coding block, The step of dividing the training sample set into M optimization sample subsets based on the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block, wherein each sample coding block is located in the optimization sample subset of the filtering model corresponding to the filter block having the largest peak signal-to-noise ratio among the M filter blocks corresponding to the sample coding block. The method according to claim 16, including the method described in claim 16.

18. A filtering device used on the encoder side, A first decision module configured to determine K filtering model groups based on K reference quantization parameters obtained from a target correspondence relationship based on the quantization parameters of a target image, wherein each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1. A second decision module configured to determine the reconstruction block corresponding to the currently coding block in the target image, A third decision module configured to determine a target filtering model from the K filtering model groups, wherein the target filtering model is a filtering model corresponding to the smallest coding distortion present after the reconstructed block has been filtered, and the coding distortion present after the reconstructed block has been filtered based on the target filtering model is smaller than the coding distortion of the reconstructed block. A first filtering module configured to filter the reconfiguration block based on the target filtering model, and A device that includes this.

19. A filtering device used on the decoder side, A first decision module configured to determine K filtering model groups, where each of the K filtering model groups contains M filtering models, the same filtering model group corresponds to the same quantization parameters, different filtering model groups correspond to different quantization parameters, and both K and M are integers greater than 1. A second decision module configured to determine the reconfiguration block based on the bitstream, A third decision module configured to determine a target filtering model from the aforementioned K filtering model groups, A filtering module configured to filter the reconfiguration block based on the target filtering model, A device that includes this.

20. A filtering model training device, An acquisition module configured to acquire a training sample set, wherein the training sample set includes a plurality of sample coding blocks and a reconstruction block corresponding to each sample coding block, and the quantization parameters of one or more images to which the plurality of sample coding blocks belong are the same. A first training module configured to train a filtering model to be trained based on the aforementioned training sample set and to obtain an initial filtering model, A splitting module configured to divide the training sample set into M initial sample subsets, each initial sample subset comprising at least two sample coding blocks and a reconfiguration block corresponding to the at least two sample coding blocks, A second training module configured to separately train the initial filtering models based on the M initial sample subsets to obtain M optimized filtering models, A third training module configured to train the M optimization filtering models based on the aforementioned training sample set to obtain one filtering model group, and A device that includes this.

21. It is an encoder-side device, The encoder-side device includes memory and a processor, An encoder-side device wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to realize the method according to claim 1 or 2.

22. Decoder-side device, The decoder-side device includes memory and a processor, A decoder-side device wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to realize the method according to claim 8 or 9.

23. A filtering model training device, The filtering model training device includes memory and a processor, A filtering model training device wherein the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to realize the method according to claim 14 or 15.

24. A computer-readable storage medium, A computer-readable storage medium that stores instructions, and when the instructions are executed on a computer, the computer is able to perform the steps of the method according to claim 1 or 8.

25. A computer program, the computer program includes instructions, and when the instructions are executed on a computer, the computer is able to perform the steps of the method according to claim 1 or 8.

Images