Filtering methods, filtering model training methods and related devices

Abstract

This application discloses a filtering method, a filtering model training method, and related apparatus, belonging to the field of encoding and decoding technology. The method includes: determining K sets of filtering models based on the quantization parameters of the target image; determining the reconstructed block corresponding to the current coded block in the target image; determining a target filtering model from the K sets of filtering models; and filtering the reconstructed block based on the target filtering model. Since the same set of filtering models is suitable for coded blocks of the same coding quality, different sets of filtering models are suitable for coded blocks of different coding qualities, and different filtering models within the same set are suitable for coded blocks with different content, for the reconstructed block corresponding to the current coded block, a target filtering model can be selected from the K sets of filtering models by combining the coding quality and the content of the coded block, and then the reconstructed block can be filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance.

Description

Technical Field

[0001] This application relates to the field of encoding and decoding technology, and in particular to a filtering method, a filtering model training method, and related apparatus. Background Technology

[0002] Encoding and decoding technologies have wide applications in multimedia services, broadcasting, video communication, and storage. During encoding, an image is divided into multiple non-overlapping coded blocks, which are then encoded sequentially. During decoding, each reconstructed block is parsed from the bitstream sequentially to determine the reconstructed image. However, in some cases, adjacent reconstructed blocks may exhibit excessive smoothness or pixel discontinuities, leading to image distortion between the reconstructed and original images. Therefore, filtering of the reconstructed blocks is necessary. Furthermore, when encoding blocks using intra-frame prediction or inter-frame prediction methods at the encoding end, filtering of the reconstructed blocks is also required to ensure the encoding quality of subsequent blocks.

[0003] In related technologies, for each of multiple quantization parameters, a filtering model is pre-trained for each quantization parameter. When filtering the reconstructed block at the encoder, filtering models corresponding to multiple quantization parameters adjacent to the image's quantization parameters are selected from these pre-trained filtering models to obtain multiple filtering models. Then, a target filtering model is selected from these multiple filtering models, and the reconstructed block is filtered using the target filtering model. Furthermore, the encoder can encode the index of the target filtering model into the bitstream and send it to the decoder. After receiving the bitstream sent by the encoder, the decoder can determine the index of the reconstructed block and the target filtering model by parsing the bitstream, and then filter the reconstructed block using the target filtering model based on the index of the target filtering model.

[0004] Since one quantization parameter corresponds to one filtering model, but different coding blocks in the same image may require different filtering, the network structure of the filtering model corresponding to each quantization parameter will be more complex in order to meet the filtering effect of each coding block in the same image. As a result, the filtering speed according to the above method will be affected, which may affect the encoding and decoding speed of the image. Summary of the Invention

[0005] This application provides a filtering method, a filtering model training method, and related apparatus, which can improve filtering performance while simplifying the network model, and meet the filtering requirements for coding blocks of different qualities and contents in the same image. The technical solution is as follows:

[0006] Firstly, a filtering method is provided for use at the encoding end. In this method, K sets of filtering models are determined based on the quantization parameters of the target image. Each set of K filtering models includes M filtering models, with the same quantization parameter corresponding to the same set of models, and different sets of models corresponding to different quantization parameters. K and M are both integers greater than 1. The reconstructed block corresponding to the current coded block in the target image is determined. A target filtering model is determined from the K sets of filtering models. The target filtering model is the filtering model that minimizes the encoding distortion after filtering the reconstructed block, and the encoding distortion of the reconstructed block after filtering with the target filtering model is less than the encoding distortion of the reconstructed block itself. The reconstructed block is then filtered based on the target filtering model.

[0007] Optionally, the encoder obtains K reference quantization parameters from the target correspondence based on the quantization parameters of the target image. Since each quantization parameter corresponds to a set of filtering models, the encoder can determine K sets of filtering models based on these K reference quantization parameters.

[0008] The target correspondence is used to indicate the correspondence between image quantization parameters and reference quantization parameters. As an example, the target correspondence is the correspondence between the quantization parameter range and the reference quantization parameters, or the target correspondence is the correspondence between image quantization parameters and reference quantization parameters.

[0009] When the target correspondence is a correspondence between quantization parameter ranges and reference quantization parameters, since the reference quantization parameters corresponding to each quantization parameter within the same quantization parameter range are identical, the encoder only needs to store the quantization parameter range and does not need to store each quantization parameter sequentially. This saves storage space on the encoder, thereby improving the efficiency of the encoder in determining the K sets of filtering models.

[0010] When the target correspondence is a correspondence between image quantization parameters and reference quantization parameters, since one image quantization parameter corresponds to K reference quantization parameters, the correlation between these K reference quantization parameters and the image quantization parameter is stronger. Therefore, the K sets of filtering models determined by the encoder according to the target correspondence have a stronger correlation with the quantization parameters of the target image, thereby further improving the filtering effect.

[0011] Since the quantization parameters corresponding to a coding block determine its coding quality—smaller quantization parameters result in higher coding quality, and larger quantization parameters result in lower coding quality—and because the same set of filtering models corresponds to the same quantization parameter, while different sets of filtering models correspond to different quantization parameters, multiple coding blocks encoded with the same quantization parameter will have the same coding quality and can be filtered by the same set of filtering models. Conversely, multiple coding blocks encoded with different quantization parameters will have different coding qualities and can be filtered by different sets of filtering models. In other words, the same set of filtering models is suitable for coding blocks of the same coding quality, while different sets of filtering models are suitable for coding blocks of different coding qualities.

[0012] Optionally, after determining the K groups of filtering models based on the quantization parameters of the target image, the encoding end also needs to encode the quantization parameters corresponding to the K groups of filtering models into the bitstream.

[0013] Optionally, the encoder determines the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models. The filtering indication information is used to indicate whether the reconstructed block needs filtering. If the filtering indication information indicates that the reconstructed block needs filtering, a target filtering model is determined from the K sets of filtering models.

[0014] The encoder inputs the reconstructed block into each of the K filtering models to obtain K*M filtering blocks. Based on the current encoded block, the reconstructed block, and the K*M filtering blocks, it determines the rate-distortion cost corresponding to the reconstructed block and the rate-distortion cost corresponding to each filtering block. If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtering block, then the filtering indication information is determined as the first indication information, which indicates that the reconstructed block needs filtering. If the rate-distortion cost corresponding to the reconstructed block is less than the rate-distortion cost corresponding to each filtering block, then the filtering indication information is determined as the second indication information, which indicates that the reconstructed block does not need filtering.

[0015] Rate-distortion cost is used to indicate the degree of image distortion between the reconstructed block and the original coded block, as well as the degree of image distortion between the filtered block and the original coded block. If the rate-distortion cost corresponding to the reconstructed block is less than the rate-distortion cost corresponding to each filtered block, it indicates that the image distortion between the reconstructed block and the original coded block is minimal. Thus, the image distortion between the reconstructed image based on this block and the original image is minimal. In this case, filtering of the reconstructed block is unnecessary. If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtered block, it indicates that the image distortion between the filtered block and the original coded block is minimal. Thus, the image distortion between the reconstructed image based on the filtered block and the original image is minimal. In this case, filtering of the reconstructed block is necessary.

[0016] Based on the above description, taking rate-distortion cost as an example to indicate coding distortion, when the filtering indication information indicates that the reconstructed block needs filtering, the rate-distortion cost corresponding to each filtering block is compared, and the filtering model corresponding to the filtering block with the lowest rate-distortion cost is determined as the target filtering model. Of course, in practical applications, coding distortion can also be indicated by other parameters, and this application embodiment does not limit this.

[0017] Optionally, after determining the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K groups of filtering models, the encoding end also needs to encode the filtering indication information into the bitstream.

[0018] Optionally, after the encoder determines the target filtering model from the K groups of filtering models, it also needs to encode the target index into the bitstream. The target index is used to indicate the target filtering model.

[0019] It should be noted that the above example assumes the filter indication information indicates that the reconstructed block needs filtering. In practical applications, the filter indication information may also indicate that the reconstructed block does not need filtering. If the filter indication information indicates that the reconstructed block does not need filtering, then filtering will not be performed on that reconstructed block.

[0020] Since the same set of filtering models is suitable for coding blocks of the same coding quality, different sets of filtering models are suitable for coding blocks of different coding quality, and different filtering models within the same set are suitable for coding blocks with different content, after determining K sets of filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K sets of filtering models by combining the coding quality and the content of the coding block. Then, filtering is performed on the reconstructed block based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Furthermore, for coding blocks of different coding quality and content within the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering requirements for coding blocks of different quality and content within the same image.

[0021] Secondly, a filtering method is provided for application at the decoding end. In this method, K groups of filtering models are determined, each group comprising M filtering models. The same group of filtering models corresponds to the same quantization parameter, while different groups of filtering models correspond to different quantization parameters. K and M are both integers greater than 1. A reconstructed block is determined based on the bitstream, a target filtering model is determined from the K groups of filtering models, and the reconstructed block is filtered based on the target filtering model.

[0022] Optionally, the decoding end determines K sets of filtering models based on the quantization parameters of the target image to which the reconstructed block belongs.

[0023] Optionally, after the encoding end determines the K groups of filtering models based on the quantization parameters of the target image, it also encodes the quantization parameters corresponding to the K groups of filtering models into the bitstream. Therefore, after receiving the bitstream, the decoding end can parse the quantization parameters corresponding to the K groups of filtering models from the bitstream, and then determine the K groups of filtering models based on the quantization parameters corresponding to the K groups of filtering models.

[0024] The decoder determines the filtering indication information for the reconstructed block, which indicates whether the reconstructed block needs filtering. If the filtering indication information indicates that the reconstructed block needs filtering, the target filtering model among the K groups of filtering models is determined.

[0025] Since the encoder determines the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models, and then encodes the filtering indication information into the bitstream, the decoder, upon receiving the bitstream, can parse the filtering indication information from it and determine whether filtering is needed for the reconstructed block. If the filtering indication information indicates that the reconstructed block needs filtering, the decoder can parse the target index from the bitstream and determine the target filtering model based on the target index.

[0026] Since the same set of filtering models is suitable for coding blocks of the same coding quality, different sets of filtering models are suitable for coding blocks of different coding quality, and different filtering models within the same set are suitable for coding blocks with different content, after determining K sets of filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K sets of filtering models by combining the coding quality and the content of the coding block. Then, filtering is performed on the reconstructed block based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Furthermore, for coding blocks of different coding quality and content within the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering requirements for coding blocks of different quality and content within the same image.

[0027] Thirdly, a method for training a filtering model is provided. In this method, a training sample set is obtained, comprising multiple sample coding blocks and a corresponding reconstruction block for each sample coding block, wherein the quantization parameters of the images to which the multiple sample coding blocks belong are the same. Based on the training sample set, a filtering model to be trained is trained to obtain an initial filtering model. The training sample set is divided into M initial sample subsets, each initial sample subset comprising at least two sample coding blocks and corresponding reconstruction blocks for the at least two sample coding blocks. Based on the M initial sample subsets, the initial filtering model is trained respectively to obtain M optimized filtering models. Based on the training sample set, the M optimized filtering models are trained to obtain a set of filtering models.

[0028] These multiple sample coding blocks are obtained by dividing the image from multiple sample images, or they can be divided from a single sample image. That is, the multiple sample coding blocks may come from the same sample image or different sample images, as long as the quantization parameters of the images to which the multiple sample coding blocks belong are the same. Since these multiple sample coding blocks are obtained by dividing the image into multiple non-overlapping coding blocks, the content of these multiple sample coding blocks is different.

[0029] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the initial filtering model to obtain the filter block corresponding to each sample coding block. Based on the multiple sample coding blocks and the filter block corresponding to each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined. The multiple sample coding blocks are sorted according to the order of the peak signal-to-noise ratio. According to the sorting result, the training sample set is divided into M initial sample subsets. Each initial sample subset includes at least two consecutive sample coding blocks in the sorting result.

[0030] Based on this training sample set, the M optimized filtering models are trained through a cyclic iterative process. The i-th iteration in this cyclic iterative process includes the following steps:

[0031] (1) Based on the multiple sample coding blocks and the reconstruction block corresponding to each sample coding block, the training sample set is divided into M optimized sample subsets. The M optimized sample subsets correspond one-to-one with the M filtering models processed in the i-th iteration. Among them, the M filtering models processed in the first iteration are the M optimized filtering models.

[0032] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the M filtering models processed in the i-th iteration to obtain the M filtering blocks corresponding to each sample coding block. Based on the multiple sample coding blocks and the M filtering blocks corresponding to each sample coding block, the peak signal-to-noise ratio (PSNR) of the M filtering blocks corresponding to each sample coding block is determined. Based on the PSNR of the M filtering blocks corresponding to each sample coding block, the training sample set is divided into the M optimized sample subsets, where each sample coding block is located in the optimized sample subset of the filtering model corresponding to the filtering block with the largest PSNR among its M filtering blocks.

[0033] (2) Based on the M optimized sample subsets, train the M filtering models processed in the i-th iteration.

[0034] (3) If i is less than the iteration number threshold, the M filter models processed in the i-th iteration after training will be used as the M filter models processed in the (i+1)-th iteration, and the (i+1)-th iteration will be executed.

[0035] (4) If i is greater than or equal to the iteration number threshold, then the M filter models processed in the i-th iteration after training are determined as a set of filter models.

[0036] This application trains M optimized filtering models through iterative iteration. When the iteration count i of the M filtering models is less than the iteration count threshold, it indicates that the currently trained optimized filtering model is unreliable. In this case, the M filtering models processed in the i-th iteration are used as the M filtering models for the (i+1)-th iteration, and the (i+1)-th iteration is performed. When the iteration count i of the M filtering models is greater than or equal to the iteration count threshold, it indicates that the currently trained optimized filtering model is reliable. Therefore, the iteration process stops, and the M filtering models processed in the i-th iteration are used as a set of filtering models.

[0037] The iteration number threshold is preset. This iteration number threshold can be a specified number of iterations or a maximum number of iterations, and can be set according to different needs. This application embodiment does not limit this.

[0038] It should be noted that during the process of dividing the training sample set into M optimized sample subsets based on the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block, there may be a case where the training sample set is divided into only one optimized sample subset. That is, the maximum PSNR among the M filter blocks corresponding to each sample coding block in the training sample set corresponds to the same filtering model. In this case, the filtering model for the i-th iteration is trained based on the one optimized sample subset, while the iteration processing of other filtering models is stopped.

[0039] In this application, since the images to which multiple sample coding blocks belong in the training sample set have the same quantization parameters, a set of filtering models is obtained by training an untrained filtering model based on this training sample set, and is applicable to coding blocks of the same coding quality. Furthermore, since the content of the multiple sample coding blocks in the training sample set is different, the M filtering models in this set are applicable to coding blocks with different content.

[0040] Fourthly, a filtering device is provided, which has the function of implementing the filtering method described in the first aspect. The filtering device includes at least one module for implementing the filtering method provided in the first aspect.

[0041] Fifthly, a filtering device is provided, which has the function of implementing the filtering method described in the second aspect above. The filtering device includes at least one module for implementing the filtering method provided in the second aspect above.

[0042] Sixthly, a filtering model training apparatus is provided, which has the function of implementing the filtering model training method described in the third aspect above. The filtering model training apparatus includes at least one module for implementing the filtering model training method provided in the third aspect above.

[0043] In a seventh aspect, an encoding end device is provided, the encoding end device including a processor and a memory, the memory being used to store a computer program for executing the filtering method provided in the first aspect. The processor is configured to execute the computer program stored in the memory to implement the filtering method described in the first aspect.

[0044] Optionally, the encoding end device may further include a communication bus for establishing a connection between the processor and the memory.

[0045] Eighthly, a decoding device is provided, comprising a processor and a memory, the memory being used to store a computer program for executing the filtering method provided in the second aspect above. The processor is configured to execute the computer program stored in the memory to implement the filtering method described in the second aspect above.

[0046] Optionally, the decoding device may further include a communication bus for establishing a connection between the processor and the memory.

[0047] A ninth aspect provides a filtering model training apparatus, the apparatus comprising a processor and a memory, the memory being used to store a computer program for executing the filtering model training method provided in the third aspect above. The processor is configured to execute the computer program stored in the memory to implement the filtering model training method described in the third aspect above.

[0048] Optionally, the filtering model training device may further include a communication bus for establishing a connection between the processor and the memory.

[0049] In a tenth aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the filtering method described in the first aspect, the steps of the filtering method described in the second aspect, or the steps of the filtering model training method described in the third aspect.

[0050] Eleventhly, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to perform the steps of the filtering method described in the first aspect, the filtering method described in the second aspect, or the filtering model training method described in the third aspect. Alternatively, a computer program is provided that, when executed on a computer, causes the computer to perform the steps of the filtering method described in the first aspect, the filtering method described in the second aspect, or the filtering model training method described in the third aspect.

[0051] The technical effects achieved by the fourth to eleventh aspects mentioned above are similar to the technical effects achieved by the corresponding technical means in the first, second or third aspects, and will not be repeated here. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0053] Figure 2 This is an exemplary structural block diagram of an encoding end provided in an embodiment of this application;

[0054] Figure 3 This is an exemplary structural block diagram of a decoding end provided in an embodiment of this application;

[0055] Figure 4 This is a flowchart of a filtering method provided in an embodiment of this application;

[0056] Figure 5 This is a flowchart of another filtering method provided in the embodiments of this application;

[0057] Figure 6 This is a flowchart of a filtering model training method provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of a filtering device provided in an embodiment of this application;

[0059] Figure 8 This is a schematic diagram of another filtering device provided in an embodiment of this application;

[0060] Figure 9 This is a schematic diagram of the structure of a filtering model training device provided in an embodiment of this application;

[0061] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0063] Before providing a detailed explanation of the filtering method provided in the embodiments of this application, the terminology and implementation environment involved in the embodiments of this application will be introduced first.

[0064] To facilitate understanding, the terms used in the embodiments of this application will be explained first.

[0065] Encoding refers to the process of compressing an image into a bitstream. The image can be a still image, a moving image, or any video frame included in a video.

[0066] Decoding refers to the process of restoring an encoded bitstream into a reconstructed image according to specific syntax rules and processing methods.

[0067] A coding block is a region of an image that has been divided into coding blocks. An image can be divided into multiple coding blocks, which together form an image. Each coding block can be encoded independently. For example, the size of this coding block is 128*128.

[0068] Quantization is the process of mapping continuous values ​​of a signal to multiple discrete amplitude values. Quantization can effectively reduce the range of signal values, thereby achieving better compression, and it is also the root cause of distortion.

[0069] The quantization parameter (QP) is a crucial parameter controlling the degree of quantization and reflects the compression level of the image. Generally, a smaller QP results in finer quantization, preserving more image details and producing higher encoded quality, but also requiring more bitrate. Conversely, a larger QP results in coarser quantization, greater loss of image details, lower encoded quality, more noticeable distortion, and less bitrate. In other words, the quantization parameter and bitrate are negatively correlated.

[0070] Intra-frame prediction refers to predicting the current coding block based on the reconstructed blocks corresponding to the coding blocks coded before it in the same image. For example, the current coding block is predicted using the reconstructed blocks corresponding to the coding blocks coded to its left and above.

[0071] Inter-frame prediction refers to determining the reconstructed image corresponding to the previously encoded image as the reference image, and predicting the current coded block based on the reconstructed blocks in the reference image that are similar to the current coded block.

[0072] The implementation environment involved in the embodiments of this application will be described next.

[0073] Encoding and decoding technologies have wide applications in multimedia services, broadcasting, video communication, and storage. During encoding, an image is divided into multiple non-overlapping coded blocks, which are then encoded sequentially. During decoding, each reconstructed block is parsed from the bitstream sequentially to determine the reconstructed image. However, in some cases, adjacent reconstructed blocks may exhibit excessive smoothness or pixel discontinuities, leading to image distortion between the reconstructed and original images. Therefore, filtering of the reconstructed blocks is necessary. Furthermore, when encoding blocks using intra-frame prediction or inter-frame prediction methods at the encoding end, filtering of the reconstructed blocks is also required to ensure the encoding quality of subsequent blocks.

[0074] Please refer to Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 is used to encode each coding block in the image, and during the encoding process according to intra-frame prediction or inter-frame prediction, it is also used to filter the reconstructed blocks of the coding blocks. The destination device 20 is used to parse the bitstream to determine the reconstructed blocks, and is also used to filter the reconstructed blocks.

[0075] Since the source device 10 is used to encode the image to generate a bitstream, it is also called an image encoding device or an image encoding end. The destination device 20 is used to decode the bitstream generated by the source device 10. Therefore, the destination device 20 is also called an image decoding device or an image decoding end.

[0076] Link 30 is used to receive the bitstream generated by source device 10 and transmit it to destination device 20. Storage device 40 is used to receive and store the bitstream generated by source device 10, under which the destination device 20 can directly obtain the bitstream from storage device 40. Alternatively, storage device 40 corresponds to a file server or another intermediate storage device capable of storing the bitstream generated by source device 10, under which the destination device 20 can stream or download the bitstream stored in storage device 40.

[0077] Both source device 10 and destination device 20 include one or more processors and memory coupled to the one or more processors, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, and any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, both source device 10 and destination device 20 include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smartphones," televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0078] Link 30 includes one or more media or devices capable of transmitting a bitstream from source device 10 to destination device 20. In one possible implementation, link 30 includes one or more communication media enabling source device 10 to directly transmit the bitstream to destination device 20 in real time. In this embodiment, source device 10 modulates the bitstream according to a communication standard, such as a wireless communication protocol, and transmits the bitstream to destination device 20. The one or more communication media include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media can form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20, etc., which are not specifically limited in this embodiment.

[0079] In one possible implementation, storage device 40 is used to store the received bitstream transmitted by source device 10, and destination device 20 can directly retrieve the bitstream from storage device 40. Under such conditions, storage device 40 includes any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memory (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing bitstreams.

[0080] In one possible implementation, storage device 40 corresponds to a file server or another intermediate storage device capable of storing the bitstream generated by source device 10, and destination device 20 can stream or download images stored on storage device 40. The file server is any type of server capable of storing and sending bitstreams to destination device 20. In one possible implementation, the file server includes a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 can acquire the bitstream via any standard data connection (including an Internet connection). Any standard data connection includes wireless channels (e.g., Wi-Fi connections), wired connections (e.g., digital subscriber line (DSL), cable modems, etc.), or combinations of both suitable for acquiring bitstreams stored on a file server. The transmission of the bitstream from storage device 40 can be streaming, downloading, or a combination of both.

[0081] Figure 1 The implementation environment shown is only one possible implementation, and the technology of this application embodiment is not only applicable to... Figure 1 The source device 10, which is capable of encoding images, and the destination device 20, which is capable of decoding bitstreams, shown are also applicable to other devices capable of encoding images and decoding bitstreams, and the embodiments of this application do not specifically limit them.

[0082] exist Figure 1 In the illustrated implementation environment, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 includes a modem / demodulator and / or a transmitter, wherein the transmitter is also referred to as a transmitter. Data source 120 includes an image capture device (e.g., a camera, etc.), an archive containing previously captured images, a feed interface for receiving images from an image content provider, and / or a computer graphics system for generating images, or a combination of these sources of images.

[0083] Data source 120 sends images to encoder 100, which encodes the received images to obtain a bitstream. The encoder sends the bitstream to an output interface. In some embodiments, source device 10 sends the bitstream directly to destination device 20 via output interface 140. In other embodiments, the bitstream may also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or display.

[0084] exist Figure 1 In the illustrated implementation environment, destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive a bitstream via link 30 and / or from storage device 40, and then send it to the decoder 200, which decodes the received bitstream to obtain a reconstructed image. The decoder sends the reconstructed image to the display device 220. The display device 220 may be integrated with destination device 20 or may be external to destination device 20. Generally, the display device 220 displays the reconstructed image. The display device 220 can be any type of display device, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0085] although Figure 1 Not shown, but in some respects, encoder 100 and decoder 200 may each be integrated with an audio encoder and decoder, and include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

[0086] Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques of the embodiments of this application are implemented in part in software, the apparatus may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the techniques of the embodiments of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors. Each of encoder 100 and decoder 200 is included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (encoder-decoder) in the respective apparatus.

[0087] In this application embodiment, encoder 100 may be generally referred to as an apparatus that “signals” or “sends” certain information to, for example, decoder 200. The terms “signals” or “sends” may generally refer to the transmission of syntax elements and / or other data used for decoding a bitstream. This transmission may occur in real-time or nearly real-time. Alternatively, this communication may occur after a period of time, for example, during encoding when syntax elements are stored in a computer-readable storage medium in the encoded bitstream, and the decoding apparatus may then retrieve the syntax elements at any time after they have been stored in this medium.

[0088] Please refer to Figure 2 , Figure 2This is an exemplary structural block diagram of an encoding end provided in an embodiment of this application. The encoding end includes a predictor, a transformer, a quantizer, an entropy encoder, an inverse quantizer, an inverse transformer, a filter, and a memory. The predictor is either an intra-frame predictor or an inter-frame predictor. That is, for the current coding block in the target image to be encoded, the encoding end can perform intra-frame prediction on the current coding block through the intra-frame predictor, and can also perform inter-frame prediction on the current coding block through the inter-frame predictor. When the encoding end performs intra-frame prediction on the current coding block, it retrieves a first reference reconstruction block from the memory, and based on the first reference reconstruction block, performs intra-frame prediction on the current coding block through the intra-frame predictor to obtain the prediction block corresponding to the current coding block. The first reference reconstruction block is the reconstruction block corresponding to the coding block that was encoded before the current coding block in the target image. Alternatively, when the encoder performs inter-frame prediction on the current coding block, it retrieves the second reference reconstruction block from the memory, and then, based on the second reconstruction block, predicts the current coding block through the inter-frame predictor to obtain the prediction block corresponding to the current coding block. The second reference reconstruction block is a reconstruction block similar to the current coding block in the previously encoded image of the target image.

[0089] After the encoder determines the prediction block corresponding to the current coding block using the intra-frame predictor or inter-frame predictor as described above, the difference between the current coding block and the prediction block is determined as the residual block. Then, the residual block is transformed by a transformer to obtain the transformed residual block, which is then quantized by a quantizer to obtain the quantized residual block. Finally, the quantized residual block and prediction indication information are encoded into the bitstream by an entropy encoder, and the bitstream is sent to the decoder. This prediction indication information indicates the prediction mode used when predicting the current coding block.

[0090] To ensure the coding quality of the next coding block adjacent to the current coding block, before encoding the next coding block at the encoding end, the residual block after quantization transformation needs to be dequantized by an inverse quantizer to obtain a transformed residual block. This transformed residual block is then inversely transformed by an inverse transformer to obtain a reconstructed residual block. The reconstructed residual block is then added to the predicted block to obtain the reconstructed block corresponding to the current coding block. After determining the reconstructed block corresponding to the current coding block, the encoding end filters the reconstructed block corresponding to the current coding block using a filter according to the filtering method provided in the embodiments of this application to obtain a filtered block corresponding to the current coding block. This filtered block is then stored in memory for encoding the next coding block.

[0091] Based on the above description, QP is a crucial parameter controlling the degree of quantization. Therefore, during the encoding process of the current coding block, the encoder needs to determine the QP corresponding to that coding block. In practical applications, the QPs corresponding to different coding blocks in the same image may be the same or different. For example, the encoder may divide the target image into multiple non-overlapping coding blocks, and for any one of these coding blocks, the QP of the target image is used as the QP corresponding to that coding block. Alternatively, the encoder may divide the target image into multiple non-overlapping coding blocks, and for any one of these coding blocks, the QP of that coding block is adaptively adjusted based on the QP of the target image.

[0092] The multiple coded blocks may be of the same size or of different sizes. That is, the encoding end divides the target image into coded blocks of the same size, or the encoding end divides the target image into coded blocks of different sizes according to the content of the target image. The shape of the coded blocks may be square or other shapes; the embodiments of this application do not limit the shape of the coded blocks.

[0093] Optionally, the transformer is any one of a discrete cosine transform (DCT), a discrete sine transform (DST), or a karhunen-loève transform (KLT).

[0094] Please refer to Figure 3 , Figure 3 This is an exemplary structural block diagram of a decoding end provided in an embodiment of this application. The decoding end includes an entropy decoder, a predictor, an inverse quantizer, an inverse transform, a memory, and a filter. The predictor is either an intra-frame predictor or an inter-frame predictor. That is, for the target image, if the encoding end performs intra-frame prediction on each coded block in the target image, the decoding end also needs to determine the prediction block using an intra-frame predictor. If the encoding end performs inter-frame prediction on each coded block in the target image, the decoding end also needs to determine the prediction block using an inter-frame predictor.

[0095] After receiving the bitstream, the decoder decodes it using an entropy decoder to obtain the quantized residual block and prediction indication information. This prediction indication information indicates the prediction mode used when predicting the current coding block. Then, based on this prediction indication information, the decoder determines whether to use an intra-frame predictor or an inter-frame predictor for prediction. If the intra-frame predictor is used, the decoder retrieves the first reference reconstructed block from memory and uses the intra-frame predictor to determine the prediction block corresponding to the current coding block. If the inter-frame predictor is used, the decoder retrieves the second reference reconstructed block from memory and uses the inter-frame predictor to determine the prediction block corresponding to the current coding block. Then, the quantized residual block is sequentially passed through an inverse quantizer and an inverse transformer to obtain a reconstructed residual block. This reconstructed residual block is then added to the prediction block to obtain the reconstructed block corresponding to the current coding block.

[0096] To avoid image distortion between the reconstructed block corresponding to the current coding block and the current coding block, and to avoid excessive unsmoothness or pixel discontinuity between adjacent reconstructed blocks, the decoding end can also filter the reconstructed block using a filter according to the filtering method provided in the embodiments of this application.

[0097] It should be noted that the business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0098] The filtering method provided in the embodiments of this application will now be explained in detail.

[0099] Figure 4 This is a flowchart of a filtering method provided in an embodiment of this application. This method is applied at the encoding end; please refer to [the documentation / reference]. Figure 4 The method includes the following steps.

[0100] Step 401: Based on the quantization parameters of the target image, determine K sets of filtering models. Each set of filtering models includes M filtering models, and the same set of filtering models corresponds to the same quantization parameter. Different sets of filtering models correspond to different quantization parameters. K and M are both integers greater than 1.

[0101] In some embodiments, the encoder obtains K reference quantization parameters from the target correspondence based on the quantization parameters of the target image. Since each quantization parameter corresponds to a set of filtering models, the encoder can determine K sets of filtering models based on these K reference quantization parameters.

[0102] The target correspondence is used to indicate the correspondence between image quantization parameters and reference quantization parameters. As an example, the target correspondence is the correspondence between the quantization parameter range and the reference quantization parameters, or the target correspondence is the correspondence between image quantization parameters and reference quantization parameters.

[0103] When the target correspondence is a correspondence between quantization parameter ranges and reference quantization parameters, the encoder first determines the quantization parameter range of the target image to obtain the target quantization parameter range. Then, based on the target quantization parameter range, it obtains the K reference quantization parameters corresponding to the target quantization parameter range from the target correspondence.

[0104] For example, the target correspondence is shown in Table 1 below. In Table 1, one quantization parameter range corresponds to three reference quantization parameters. Table 1 is an example where each quantization parameter range corresponds to three reference quantization parameters, meaning the number of reference quantization parameters corresponding to each quantization parameter range is the same. Of course, in practical applications, the number of reference quantization parameters corresponding to each quantization parameter range may not be the same.

[0105] Table 1

[0106] Quantization parameter range Reference quantization parameters QP≤24 QP=22, QP=17, QP=27 24<QP≤29 QP=27, QP=22, QP=17 29<QP≤34 QP=32, QP=27, QP=22 34<QP≤39 QP=42, QP=37, QP=32

[0107] When the target correspondence is a correspondence between image quantization parameters and reference quantization parameters, the encoder directly obtains the K reference quantization parameters corresponding to the quantization parameters of the target image from the target correspondence based on the quantization parameters of the target image.

[0108] For example, the target correspondence is shown in Table 2 below. In Table 2, one image quantization parameter corresponds to three reference quantization parameters. Table 2 is an example where each image quantization parameter corresponds to three reference quantization parameters, meaning the number of reference quantization parameters is the same for each image quantization parameter. Of course, in practical applications, the number of reference quantization parameters corresponding to each image quantization parameter may not be the same.

[0109] Table 2

[0110] Image quantization parameters Reference quantization parameters QP = 24 QP=22, QP=17, QP=27 QP = 29 QP=27, QP=22, QP=17 QP=34 QP=32, QP=27, QP=22 QP=39 QP=42, QP=37, QP=32

[0111] When the target correspondence is a correspondence between quantization parameter ranges and reference quantization parameters, since the reference quantization parameters corresponding to each quantization parameter within the same quantization parameter range are identical, the encoder only needs to store the quantization parameter range and does not need to store each quantization parameter sequentially. This saves storage space on the encoder, thereby improving the efficiency of the encoder in determining the K sets of filtering models.

[0112] When the target correspondence is a correspondence between image quantization parameters and reference quantization parameters, since one image quantization parameter corresponds to K reference quantization parameters, the correlation between these K reference quantization parameters and the image quantization parameter is stronger. Therefore, the K sets of filtering models determined by the encoder according to the target correspondence have a stronger correlation with the quantization parameters of the target image, thereby further improving the filtering effect.

[0113] Since the quantization parameters corresponding to a coding block determine its coding quality—smaller quantization parameters result in higher coding quality, and larger quantization parameters result in lower coding quality—and because the same set of filtering models corresponds to the same quantization parameter, while different sets of filtering models correspond to different quantization parameters, multiple coding blocks encoded with the same quantization parameter will have the same coding quality and can be filtered by the same set of filtering models. Conversely, multiple coding blocks encoded with different quantization parameters will have different coding qualities and can be filtered by different sets of filtering models. In other words, the same set of filtering models is suitable for coding blocks of the same coding quality, while different sets of filtering models are suitable for coding blocks of different coding qualities.

[0114] In some embodiments, after determining the K groups of filtering models based on the quantization parameters of the target image, the encoding end also needs to encode the quantization parameters corresponding to the K groups of filtering models into the bitstream. In this way, after receiving the bitstream, the decoding end can parse the quantization parameters corresponding to the K groups of filtering models from the bitstream and determine the K groups of filtering models based on these quantization parameters.

[0115] The structure of the filtering model may be a convolutional neural network (CNN) structure, or it may be other structures. This application does not limit the structure of the filtering model.

[0116] Step 402: Determine the reconstructed block corresponding to the current coded block in the target image.

[0117] The process by which the encoder determines the reconstructed block corresponding to the current coded block in the target image can be referred to the above. Figure 2 The relevant descriptions in the document will not be repeated here.

[0118] Step 403: Determine the target filtering model from the K groups of filtering models. The target filtering model is the filtering model that minimizes the coding distortion after filtering the reconstructed block, and the coding distortion of the reconstructed block after filtering it through the target filtering model is less than the coding distortion of the reconstructed block.

[0119] In some embodiments, the encoder can determine the target filtering model from the K groups of filtering models according to the following steps (1)-(2).

[0120] (1) Based on the current coding block, the reconstruction block and the K groups of filtering models, determine the filtering indication information corresponding to the reconstruction block. The filtering indication information is used to indicate whether the reconstruction block needs to be filtered.

[0121] The encoder inputs the reconstructed block into each of the K filtering models to obtain K*M filtering blocks. Based on the current encoded block, the reconstructed block, and the K*M filtering blocks, it determines the rate-distortion cost corresponding to the reconstructed block and the rate-distortion cost corresponding to each filtering block. If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtering block, then the filtering indication information is determined as the first indication information, which indicates that the reconstructed block needs filtering. If the rate-distortion cost corresponding to the reconstructed block is less than the rate-distortion cost corresponding to each filtering block, then the filtering indication information is determined as the second indication information, which indicates that the reconstructed block does not need filtering.

[0122] As an example, the rate-distortion cost corresponding to the reconstructed block can be determined according to the following formula (1).

[0123] J=D+λR (1)

[0124] In formula (1) above, J represents the rate-distortion cost, D represents the error between the pixel value of the pixel in the reconstructed block and the pixel value of the pixel in the current coding block, λ represents the distortion parameter, which is usually the default value, and R represents the number of bits required to encode the current coding block into the bitstream when filtering of the reconstructed block is not required. Of course, in practical applications, the rate-distortion cost corresponding to each filtering block can also be determined according to formula (1) above. In this case, D in formula (1) above represents the error between the pixel value of the pixel in the filtering block and the pixel value of the pixel in the current coding block, and R represents the number of bits required to encode the current coding block into the bitstream when filtering of the reconstructed block is required.

[0125] When filtering is not required for the reconstructed block, the number of bits required to encode the current coded block into the bitstream includes the number of bits required for encoding the filtering indication information, the number of bits required for encoding the residual block after quantization transformation, and the number of bits required for encoding the prediction indication information. When filtering is required for the reconstructed block, the number of bits required to encode the current coded block into the bitstream includes the number of bits required for encoding the filtering indication information, the number of bits required for encoding the residual block after quantization transformation, the number of bits required for encoding the prediction indication information, and the number of bits required for encoding the filtering model index.

[0126] In some embodiments, the encoder stores a correspondence between the filter model index and the number of bits required to encode the filter model index. Therefore, after the encoder determines the K*M filter models, it can obtain the number of bits required to encode the K*M filter model indexes from the stored correspondence between the filter model indexes and the number of bits required to encode the filter model indexes based on the model indexes of the K*M filter models, and then determine the rate-distortion cost corresponding to each filter block according to the above formula (1).

[0127] The above content uses the example of different number of bits required to encode different filter model indices, that is, different filter model indices correspond to different number of encoding bits. Of course, in practical applications, the number of bits required to encode different filter model indices may also be the same, that is, different filter model indices correspond to the same number of encoding bits. Thus, when determining the rate-distortion cost corresponding to each filter block according to the above formula (1), the rate-distortion cost corresponding to the K*M filter blocks mainly depends on the pixel error between the K*M filter blocks and the current encoding block.

[0128] It should be noted that when determining the rate-distortion cost corresponding to the reconstructed block and the rate-distortion cost corresponding to each filter block according to the above formula (1), the error between the pixel value of the pixel in the reconstructed block and the pixel value of the pixel in the current coding block, and the error between the pixel value of the pixel in the K*M filter blocks and the pixel value of the pixel in the current coding block are any one of the sum of absolute differences (SAD), sum of absolute transformed differences (SATD), or mean squared error (MSE).

[0129] Rate-distortion cost is used to indicate the degree of image distortion between the reconstructed block and the original coded block, as well as the degree of image distortion between the filtered block and the original coded block. If the rate-distortion cost corresponding to the reconstructed block is less than the rate-distortion cost corresponding to each filtered block, it indicates that the image distortion between the reconstructed block and the original coded block is minimal. Thus, the image distortion between the reconstructed image based on this block and the original image is minimal. In this case, filtering of the reconstructed block is unnecessary. If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtered block, it indicates that the image distortion between the filtered block and the original coded block is minimal. Thus, the image distortion between the reconstructed image based on the filtered block and the original image is minimal. In this case, filtering of the reconstructed block is necessary.

[0130] The first and second indication information can take various forms, such as numerical values, characters, etc. When both the first and second indication information are numerical values, the first indication information is 0 and the second indication information is 1. Of course, the first and second indication information can also be reversed or be other numerical values; this application does not limit this.

[0131] In some embodiments, after the encoder determines the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models, it also needs to encode the filtering indication information into the bitstream. In this way, after receiving the bitstream, the decoder can determine whether the reconstructed block needs to be filtered based on the bitstream.

[0132] (2) When the filtering indication information indicates that the reconstruction block needs to be filtered, the target filtering model is determined from the K groups of filtering models.

[0133] Based on the above description, taking rate-distortion cost as an example to indicate coding distortion, when the filtering indication information indicates that the reconstructed block needs filtering, the rate-distortion cost corresponding to each filtering block is compared, and the filtering model corresponding to the filtering block with the lowest rate-distortion cost is determined as the target filtering model. Of course, in practical applications, coding distortion can also be indicated by other parameters, and this application embodiment does not limit this.

[0134] In some embodiments, after the encoder determines the target filtering model from the K sets of filtering models, it also needs to encode the target index into the bitstream. The target index is used to indicate the target filtering model. In this way, after receiving the bitstream, the decoder can determine the target filtering model for filtering the reconstructed block based on the bitstream.

[0135] As an example, to distinguish different filtering models, each filtering model corresponds to a model index, and different filtering models correspond to different model indices. In this case, the target index includes the target model index, which is used to indicate the target filtering model in the K groups of filtering models.

[0136] As another example, since the same set of filtering models corresponds to the same quantization parameter, while different sets of filtering models correspond to different quantization parameters, to distinguish between different filtering models, the same set of filtering models corresponds to the same quality index, while different sets of filtering models correspond to different quality indices. Different filtering models within the same set of filtering models correspond to different content indices, while filtering models in different sets may have the same content index. In this case, the target index includes a target quality index and a target content index. The target quality index indicates the set of filtering models to which the target filtering model belongs, and the target content index indicates which model within that set the target filtering model is.

[0137] It should be noted that the above example assumes the filter indication information indicates that the reconstructed block needs filtering. In practical applications, the filter indication information may also indicate that the reconstructed block does not need filtering. If the filter indication information indicates that the reconstructed block does not need filtering, then filtering will not be performed on that reconstructed block.

[0138] Step 404: Filter the reconstructed block based on the target filtering model.

[0139] The reconstructed block is input into the target filtering model, which outputs a filter block according to the relevant algorithm, thereby filtering the reconstructed block.

[0140] In this embodiment, each filtering model group includes M filtering models. The same filtering model group corresponds to the same quantization parameter, while different filtering models group correspond to different quantization parameters. That is, the same filtering model group is suitable for coding blocks of the same coding quality, while different filtering models group are suitable for coding blocks of different coding qualities. Furthermore, different filtering models within the same filtering model group are suitable for coding blocks with different content. Thus, after determining K filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different content in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different content in the same image.

[0141] Figure 5 This is a flowchart of another filtering method provided in an embodiment of this application. This method is applied at the decoding end; please refer to [the documentation / reference]. Figure 5 The method includes the following steps.

[0142] Step 501: Determine K groups of filtering models. Each group of filtering models in the K groups includes M filtering models. The same group of filtering models corresponds to the same quantization parameter, and different groups of filtering models correspond to different quantization parameters. K and M are both integers greater than 1.

[0143] In some embodiments, the decoding end determines K sets of filtering models based on the quantization parameters of the target image to which the reconstructed block belongs. For detailed implementation process, please refer to the relevant description of step 401 above, which will not be repeated here.

[0144] In other embodiments, after the encoding end determines the K groups of filtering models based on the quantization parameters of the target image, it also encodes the quantization parameters corresponding to the K groups of filtering models into the bitstream. Therefore, after receiving the bitstream, the decoding end can parse the quantization parameters corresponding to the K groups of filtering models from the bitstream, and then determine the K groups of filtering models based on the quantization parameters corresponding to the K groups of filtering models.

[0145] Step 502: Determine the reconstructed blocks based on the bitstream.

[0146] After receiving the bitstream, the decoder parses the reconstructed block corresponding to the current coded block from it. The process of the decoder parsing the reconstructed block from the bitstream can be referred to the above. Figure 3 The relevant descriptions in the document will not be repeated here.

[0147] Step 503: Determine the target filtering model in the K groups of filtering models.

[0148] In some embodiments, the decoder determines filtering indication information for the reconstructed block, which indicates whether the reconstructed block needs filtering. If the filtering indication information indicates that the reconstructed block needs filtering, a target filtering model is determined from the K groups of filtering models.

[0149] Since the encoder determines the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models, and then encodes the filtering indication information into the bitstream, the decoder, upon receiving the bitstream, can parse the filtering indication information from it and then determine whether filtering is needed for the reconstructed block based on the filtering indication information.

[0150] When the filtering indication information indicates that the reconstructed block needs filtering, the encoder, after determining the target filtering model from the K sets of filtering models, will also encode the target index used to indicate the target filtering model into the bitstream. Therefore, the decoder can also parse the target index from the bitstream and then determine the target filtering model based on the target index.

[0151] Based on the above description, the target index includes the target model index, or, it includes both the target quality index and the target content index. The process by which the decoder determines the target filtering model based on the target index varies depending on the specific circumstances; therefore, the following sections will explain these two scenarios separately.

[0152] In the first case, the target index includes the target model index. In this case, the decoder directly selects the corresponding filter model from the K groups of filter models based on the target model index, and determines the selected filter model as the target filter model.

[0153] In the second scenario, the target index includes both a target quality index and a target content index. In this case, the decoder first selects a corresponding set of filter models from the K sets of filter models based on the target quality index. Then, based on the target content index, it determines the filter model corresponding to the target content index from the selected set of filter models to obtain the target filter model.

[0154] Step 504: Filter the reconstructed block based on the target filtering model.

[0155] The reconstructed block is input into the target filtering model, which outputs a filter block according to the relevant algorithm, thereby filtering the reconstructed block.

[0156] In this embodiment, each filtering model group includes M filtering models. The same filtering model group corresponds to the same quantization parameter, while different filtering models group correspond to different quantization parameters. That is, the same filtering model group is suitable for coding blocks of the same coding quality, while different filtering models group are suitable for coding blocks of different coding qualities. Furthermore, different filtering models within the same filtering model group are suitable for coding blocks with different content. Thus, after determining K filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different content in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different content in the same image.

[0157] Before filtering the reconstructed block at the encoding and decoding ends according to the above steps, it is necessary to train the untrained filtering model to obtain a set of filtering models corresponding to a quantization parameter. This set of filtering models includes M filtering models. Figure 6 This is a flowchart of a filtering model training method provided in an embodiment of this application. Please refer to it. Figure 6 The method includes the following steps.

[0158] Step 601: Obtain a training sample set, which includes multiple sample coding blocks and a reconstruction block corresponding to each sample coding block. The quantization parameters of the images to which the multiple sample coding blocks belong are the same quantization parameter.

[0159] These multiple sample coding blocks are obtained by dividing the image from multiple sample images, or they can be divided from a single sample image. That is, the multiple sample coding blocks may come from the same sample image or different sample images, as long as the quantization parameters of the images to which the multiple sample coding blocks belong are the same. Since these multiple sample coding blocks are obtained by dividing the image into multiple non-overlapping coding blocks, the content of these multiple sample coding blocks is different.

[0160] The process of obtaining the reconstructed block corresponding to each sample encoding block can be referred to the above. Figure 2 The relevant descriptions of how the encoding end determines the reconstructed block corresponding to the current encoding block will not be repeated here.

[0161] Step 602: Based on the training sample set, train the filtering model to be trained to obtain the initial filtering model.

[0162] The reconstructed blocks corresponding to multiple sample encoding blocks included in the training sample set are used as the input of the filtering model to be trained, and the multiple sample encoding blocks are used as the output of the filtering model to be trained. The filtering model to be trained is then trained to obtain the initial filtering model.

[0163] Step 603: Divide the training sample set into M initial sample subsets, each initial sample subset including at least two sample coding blocks and the corresponding reconstruction blocks of the at least two sample coding blocks.

[0164] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the initial filtering model to obtain the filter block corresponding to each sample coding block. Based on the multiple sample coding blocks and the filter block corresponding to each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined. The multiple sample coding blocks are sorted according to the order of the peak signal-to-noise ratio. According to the sorting result, the training sample set is divided into M initial sample subsets. Each initial sample subset includes at least two consecutive sample coding blocks in the sorting result.

[0165] For any sample coding block among the multiple sample coding blocks, the peak signal-to-noise ratio of the filter block corresponding to the sample coding block is determined according to the following formula (2).

[0166]

[0167] In the above formula (2), PSNR represents the peak signal-to-noise ratio of the filter block corresponding to the sample coding block, n represents the number of bits required to encode each pixel in the sample coding block, which is usually 8, and MSE represents the mean square error between the pixel value of the pixel in the sample coding block and the pixel value of the corresponding filter block.

[0168] For example, based on the sorting result of the multiple sample coding blocks, the training sample set is divided into M initial sample subsets on average, with each initial sample subset containing the same number of sample coding blocks. Of course, in practical applications, after sorting the multiple sample coding blocks according to the peak signal-to-noise ratio, the training sample set can also be divided into M initial sample subsets according to other criteria, and this embodiment does not limit this.

[0169] For example, the training sample set includes 16 sample coding blocks and a corresponding reconstruction block for each sample coding block. Assume that the 16 sample coding blocks are B0-B15, and the corresponding reconstruction blocks are C0-C15. Input the 16 reconstruction blocks C0-C15 into the initial filtering model to obtain 16 filtering blocks L0-L15. Then, determine the peak signal-to-noise ratio (PSNR) of L0-L15 according to the above formula (2) to obtain 16 PSNRs PSNR0-PSNR15. Then, sort B0-B15 according to the size order of PSNR0-PSNR15, and divide B0-B15 into 4 initial sample subsets according to the sorting result of B0-B15. Each initial sample subset includes 4 sample coding blocks.

[0170] It should be noted that sorting the multiple sample coding blocks according to their peak signal-to-noise ratio (PSNR) to divide the training sample set into M initial sample subsets is one example. In other embodiments, the training sample set can be divided into M initial sample subsets in other ways. For example, the pixel mean corresponding to each sample coding block can be determined, where the pixel mean is the average pixel value of the pixels in that sample coding block. Then, the multiple sample coding blocks can be sorted according to their pixel mean, and the training sample set can be divided into M initial sample subsets based on the sorting result. Another example is determining the pixel variance corresponding to each sample coding block, where the pixel variance is the variance of the pixel values ​​in that sample coding block. Then, the multiple sample coding blocks can be sorted according to their pixel variance, and the training sample set can be divided into M initial sample subsets based on the sorting result.

[0171] Step 604: Based on the M initial sample subsets, train the initial filtering model respectively to obtain M optimized filtering models.

[0172] For any initial sample subset among the M initial sample subsets, the reconstructed blocks corresponding to at least two sample coding blocks included in the initial sample subset are used as the input of the initial filtering model, and the at least two sample coding blocks are used as the output of the initial filtering model. The initial filtering model is then trained to obtain an optimized filtering model. In this way, for each initial sample subset among the M initial sample subsets, the initial filtering model can be trained according to the above steps to obtain M optimized filtering models.

[0173] Based on the above description, assuming that the training sample set B0-B15 is divided into four initial sample subsets, then by training the initial filtering model on these four initial sample subsets, we can obtain four optimized filtering models: filtering model A, filtering model B, filtering model C, and filtering model D.

[0174] Step 605: Based on the training sample set, train the M optimized filtering models to obtain a set of filtering models.

[0175] Based on this training sample set, the M optimized filtering models are trained through a cyclic iterative process. The i-th iteration in this cyclic iterative process includes the following steps:

[0176] (1) Based on the multiple sample coding blocks and the reconstruction block corresponding to each sample coding block, the training sample set is divided into M optimized sample subsets. The M optimized sample subsets correspond one-to-one with the M filtering models processed in the i-th iteration. Among them, the M filtering models processed in the first iteration are the M optimized filtering models.

[0177] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the M filtering models processed in the i-th iteration to obtain the M filtering blocks corresponding to each sample coding block. Based on the multiple sample coding blocks and the M filtering blocks corresponding to each sample coding block, the peak signal-to-noise ratio (PSNR) of the M filtering blocks corresponding to each sample coding block is determined. Based on the PSNR of the M filtering blocks corresponding to each sample coding block, the training sample set is divided into the M optimized sample subsets, where each sample coding block is located in the optimized sample subset of the filtering model corresponding to the filtering block with the largest PSNR among its M filtering blocks.

[0178] The process of determining the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block can be referred to the relevant description of determining the PSNR according to formula (2) in step 603 above, and will not be repeated here. For any sample coding block among the multiple sample coding blocks, determine the filter model corresponding to the maximum PSNR among the M filter blocks corresponding to the sample coding block, and then divide the sample coding block into the optimized sample subset corresponding to the filter model.

[0179] Based on the above description, the training sample set includes 16 sample coding blocks and a corresponding reconstructed block for each sample coding block. Assume that the reconstructed blocks corresponding to these 16 sample coding blocks are C0-C15. Taking reconstructed block C0 as an example, reconstructed block C0 is input into the four filtering models in the i-th iteration. The four filtering models output the four filtering blocks corresponding to sample coding block B0 as L0. A L0 B L0 C and L0 D Assume that the peak signal-to-noise ratios (PSNR) of the four peak values ​​corresponding to the sample coding block B0 are determined to be PSNR0 according to the above formula (2). A PSNR0 B PSNR0 C and PSNR0 D Among them, the peak signal-to-noise ratio (PSNR) is 0. C If the maximum value is found, then the sample coding block B0 will be assigned to the optimized sample subset corresponding to the filtering model C.

[0180] (2) Based on the M optimized sample subsets, train the M filtering models processed in the i-th iteration.

[0181] For any optimized sample subset in the M optimized sample subsets, the reconstructed block corresponding to the sample coding block included in the optimized sample subset is used as the input of the corresponding filtering model, and the sample coding block is used as the output of the corresponding filtering model to train the corresponding filtering model.

[0182] (3) If i is less than the iteration number threshold, the M filter models processed in the i-th iteration after training will be used as the M filter models processed in the (i+1)-th iteration, and the (i+1)-th iteration will be executed.

[0183] (4) If i is greater than or equal to the iteration number threshold, then the M filter models processed in the i-th iteration after training are determined as a set of filter models.

[0184] In this embodiment, the M optimized filtering models are trained using an iterative process. When the iteration count i of the M filtering models is less than the iteration count threshold, it indicates that the currently trained optimized filtering model is unreliable. The M filtering models processed in the i-th iteration are then used as the M filtering models for the (i+1)-th iteration, and the (i+1)-th iteration is performed. When the iteration count i of the M filtering models is greater than or equal to the iteration count threshold, it indicates that the currently trained optimized filtering model is reliable. The iteration process is then stopped, and the M filtering models processed in the i-th iteration are used as a set of filtering models.

[0185] The iteration number threshold is preset. This iteration number threshold can be a specified number of iterations or a maximum number of iterations, and can be set according to different needs. This application embodiment does not limit this.

[0186] It should be noted that during the process of dividing the training sample set into M optimized sample subsets based on the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block, there may be a case where the training sample set is divided into only one optimized sample subset. That is, the maximum PSNR among the M filter blocks corresponding to each sample coding block in the training sample set corresponds to the same filtering model. In this case, the filtering model for the i-th iteration is trained based on the one optimized sample subset, while the iteration processing of other filtering models is stopped.

[0187] In this embodiment, since the images to which multiple sample coding blocks belong in the training sample set have the same quantization parameters, a set of filtering models is obtained by training an untrained filtering model based on this training sample set, and is applicable to coding blocks of the same coding quality. Furthermore, since the contents of the multiple sample coding blocks in the training sample set are different, the M filtering models in this set are applicable to coding blocks with different contents. Thus, after determining K sets of filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from the K sets of filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different contents in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different contents in the same image.

[0188] Figure 7 This is a schematic diagram of a filtering device provided in an embodiment of this application. The filtering device can be implemented as part or all of an encoding end device by software, hardware, or a combination of both. The encoding end device can be... Figure 1 The source device is shown. See also Figure 7 The device includes: a first determining module 701, a second determining module 702, a third determining module 703, and a first filtering module 704.

[0189] The first determining module 701 is used to determine K groups of filtering models based on the quantization parameters of the target image. Each group of filtering models includes M filtering models, and the same group of filtering models corresponds to the same quantization parameter. Different groups of filtering models correspond to different quantization parameters, and K and M are both integers greater than 1. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0190] The second determining module 702 is used to determine the reconstructed block corresponding to the current coded block in the target image. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0191] The third determining module 703 is used to determine the target filtering model from the K groups of filtering models. The target filtering model is the filtering model that minimizes the encoding distortion after filtering the reconstructed block, and the encoding distortion of the reconstructed block after filtering with the target filtering model is less than the encoding distortion of the reconstructed block. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0192] The first filtering module 704 is used to filter the reconstructed block based on the target filtering model. For detailed implementation details, please refer to the corresponding content in the above embodiments; they will not be repeated here.

[0193] Optionally, the third determining module 703 includes:

[0194] The first determining unit is used to determine the filtering indication information corresponding to the reconstruction block based on the current coding block, the reconstruction block and the K groups of filtering models. The filtering indication information is used to indicate whether the reconstruction block needs to be filtered.

[0195] The second determining unit is used to determine the target filtering model from the K groups of filtering models when the filtering indication information indicates that the reconstructed block needs filtering.

[0196] Optionally, the first determining unit is specifically used for:

[0197] The reconstructed block is input into each of the K filtering models to obtain K*M filtering blocks;

[0198] Based on the current coding block, the reconstructed block, and the K*M filtering blocks, determine the rate-distortion cost corresponding to the reconstructed block, and the rate-distortion cost corresponding to each filtering block;

[0199] If the rate-distortion cost corresponding to the reconstruction block is not less than the rate-distortion cost corresponding to each filter block, then the filtering indication information is determined to be the first indication information, which is used to indicate that the reconstruction block needs to be filtered.

[0200] If the rate-distortion cost corresponding to the reconstruction block is less than the rate-distortion cost corresponding to each filter block, then the filter indication information is determined to be the second indication information, which is used to indicate that the reconstruction block does not need to be filtered.

[0201] Optionally, the device further includes:

[0202] The second filtering module is used to prevent filtering of the reconstructed block when the filtering indication information indicates that the reconstructed block does not need filtering.

[0203] Optionally, the third determining module 703 further includes:

[0204] The encoding unit is used to encode the filtering instruction information into the bit stream.

[0205] Optionally, the device further includes:

[0206] The first encoding module is used to encode the target index into the bitstream, and the target index is used to indicate the target filtering model.

[0207] Optionally, the device further includes:

[0208] The second encoding module is used to encode the quantization parameters corresponding to the K groups of filtering models into the bitstream.

[0209] In this embodiment, each filtering model group includes M filtering models. The same filtering model group corresponds to the same quantization parameter, while different filtering models group correspond to different quantization parameters. That is, the same filtering model group is suitable for coding blocks of the same coding quality, while different filtering models group are suitable for coding blocks of different coding qualities. Furthermore, different filtering models within the same filtering model group are suitable for coding blocks with different content. Thus, after determining K filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different content in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different content in the same image.

[0210] It should be noted that the filtering device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the filtering device and the filtering method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0211] Figure 8 This is a schematic diagram of another filtering device provided in an embodiment of this application. This filtering device can be implemented by software, hardware, or a combination of both, forming part or all of the decoding end device. The decoding end device can be... Figure 1 The target device is shown. See also Figure 8The device includes: a first determining module 801, a second determining module 802, a third determining module 803, and a filtering module 804.

[0212] The first determining module 801 is used to determine K groups of filtering models. Each group of filtering models includes M filtering models, and the same group of filtering models corresponds to the same quantization parameter. Different groups of filtering models correspond to different quantization parameters, where K and M are both integers greater than 1. For detailed implementation details, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0213] The second determining module 802 is used to determine the reconstructed block based on the bitstream. For detailed implementation process, please refer to the corresponding content in the above embodiments; it will not be repeated here.

[0214] The third determining module 803 is used to determine the target filtering model among the K groups of filtering models. For detailed implementation process, please refer to the corresponding content in the above embodiments; it will not be repeated here.

[0215] The filtering module 804 is used to filter the reconstructed block based on the target filtering model. For detailed implementation details, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0216] Optionally, the first determining module 801 is specifically used for:

[0217] Based on the quantization parameters of the target image to which the reconstructed block belongs, the K groups of filtering models are determined.

[0218] Optionally, the first determining module 801 is specifically used for:

[0219] Parse the quantization parameters corresponding to the K groups of filtering models from the bitstream;

[0220] Based on the quantization parameters corresponding to the K groups of filtering models, the K groups of filtering models are determined.

[0221] Optionally, the third determining module 803 includes:

[0222] The first determining unit is used to determine the filtering indication information of the reconstructed block, and the filtering indication information is used to indicate whether the reconstructed block needs to be filtered.

[0223] The second determining unit is used to determine the target filtering model among the K groups of filtering models when the filtering indication information indicates that the reconstruction block needs filtering.

[0224] Optionally, the first determining unit is specifically used for:

[0225] Extract the filtering indication information from the bitstream.

[0226] Optionally, the third determining module 803 is specifically used for:

[0227] The target index is parsed from the bitstream; the target index is used to indicate the target filtering model.

[0228] The target filtering model is determined based on the target index.

[0229] In this embodiment, each filtering model group includes M filtering models. The same filtering model group corresponds to the same quantization parameter, while different filtering models group correspond to different quantization parameters. That is, the same filtering model group is suitable for coding blocks of the same coding quality, while different filtering models group are suitable for coding blocks of different coding qualities. Furthermore, different filtering models within the same filtering model group are suitable for coding blocks with different content. Thus, after determining K filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from these K filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different content in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different content in the same image.

[0230] It should be noted that the filtering device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the filtering device and the filtering method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0231] Figure 9 This is a schematic diagram of a filtering model training device provided in an embodiment of this application. This filtering model training device can be implemented as part or all of a filtering model training equipment by software, hardware, or a combination of both. See also... Figure 9 The device includes: an acquisition module 901, a first training module 902, a division module 903, a second training module 904, and a third training module 905.

[0232] The acquisition module 901 is used to acquire a training sample set, which includes multiple sample coding blocks and a corresponding reconstruction block for each sample coding block. The quantization parameters of the images to which the multiple sample coding blocks belong are the same quantization parameter. For detailed implementation details, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0233] The first training module 902 is used to train the filtering model to be trained based on the training sample set to obtain an initial filtering model. For detailed implementation details, please refer to the corresponding content in the above embodiments; these will not be repeated here.

[0234] The partitioning module 903 is used to divide the training sample set into M initial sample subsets, each initial sample subset including at least two sample coding blocks and the corresponding reconstructed blocks of the at least two sample coding blocks. For detailed implementation details, please refer to the corresponding contents in the above embodiments, which will not be repeated here.

[0235] The second training module 904 is used to train the initial filtering model based on the M initial sample subsets to obtain M optimized filtering models. For detailed implementation details, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0236] The third training module 905 is used to train the M optimized filtering models based on the training sample set to obtain a set of filtering models. For detailed implementation details, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0237] Optionally, the partitioning module 903 is specifically used for:

[0238] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the initial filtering model to obtain the filtering block corresponding to each sample coding block;

[0239] Based on the multiple sample coding blocks and the corresponding filter block for each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined;

[0240] The multiple sample coding blocks are sorted according to their peak signal-to-noise ratio (PSNR).

[0241] Based on the sorting results, the training sample set is divided into M initial sample subsets, each of which includes at least two consecutive sample coding blocks from the sorting results.

[0242] Optionally, the third training module 905 is specifically used for:

[0243] Based on this training sample set, the M optimized filtering models are trained through iterative processing. The i-th iteration in this iterative process includes the following steps:

[0244] Based on the multiple sample coding blocks and the reconstruction block corresponding to each sample coding block, the training sample set is divided into M optimized sample subsets. These M optimized sample subsets correspond one-to-one with the M filtering models processed in the i-th iteration. Among them, the M filtering models processed in the first iteration are the M optimized filtering models.

[0245] Based on the M optimized sample subsets, the M filtering models processed in the i-th iteration are trained;

[0246] If i is less than the iteration number threshold, then the M filter models processed in the i-th iteration are used as the M filter models processed in the (i+1)-th iteration, and the (i+1)-th iteration is executed.

[0247] If i is greater than or equal to the iteration number threshold, then the M filter models processed in the i-th iteration after training are determined as a set of filter models.

[0248] Optionally, the third training module 905 is specifically used for:

[0249] The reconstructed blocks corresponding to the multiple sample coding blocks are input into the M filtering models processed in the i-th iteration to obtain the M filtering blocks corresponding to each sample coding block;

[0250] Based on the multiple sample coding blocks and the M filter blocks corresponding to each sample coding block, the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined;

[0251] Based on the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block, the training sample set is divided into the M optimized sample subsets. Each sample coding block is located in the optimized sample subset of the filter model corresponding to the filter block with the largest PSNR among its M filter blocks.

[0252] In this embodiment, since the images to which multiple sample coding blocks belong in the training sample set have the same quantization parameters, a set of filtering models is obtained by training an untrained filtering model based on this training sample set, and is applicable to coding blocks of the same coding quality. Furthermore, since the contents of the multiple sample coding blocks in the training sample set are different, the M filtering models in this set are applicable to coding blocks with different contents. Thus, after determining K sets of filtering models based on the quantization parameters of the target image, for the reconstructed block corresponding to the current coding block, a target filtering model can be selected from the K sets of filtering models by combining the coding quality and the content of the coding block. Then, the reconstructed block is filtered based on the target filtering model, thereby reducing coding distortion and improving filtering performance. Moreover, for coding blocks of different coding qualities and with different contents in the same image, filtering performance can be improved while simplifying the network model, satisfying the filtering effects for coding blocks of different qualities and with different contents in the same image.

[0253] It should be noted that the filtering model training device provided in the above embodiments is only illustrated by the division of the above functional modules when training the filtering model. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the filtering model training device and the filtering model training method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0254] Figure 10 This is a schematic block diagram of a computer device 1000 used in an embodiment of this application. The computer device 1000 may include a processor 1001, a memory 1002, and a bus system 1003. The processor 1001 and the memory 1002 are connected via the bus system 1003. The memory 1002 stores instructions, and the processor 1001 executes the instructions stored in the memory 1002 to perform the filtering method and filtering model training method described in the embodiments of this application. To avoid repetition, detailed descriptions are omitted here.

[0255] In this embodiment, the processor 1001 may be a central processing unit (CPU), or it may be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0256] The memory 1002 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1002. Memory 1002 may include code and data 10021 accessed by processor 1001 using bus 1003. Memory 1002 may further include an operating system 10023 and an application program 10022, which includes at least one program that allows processor 1001 to execute the filtering method or filtering model training method described in the embodiments of this application. For example, application program 10022 may include applications 1 to N, which further include applications that execute the filtering method or filtering model training method described in the embodiments of this application.

[0257] In addition to the data bus, the bus system 1003 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1003 in the figure.

[0258] Optionally, the computer device 1000 may also include one or more output devices, such as a display 1004. In one example, the display 1004 may be a haptic display, which combines a display with a haptic unit capable of operatively sensing touch input. The display 1004 may be connected to the processor 1001 via a bus 1003.

[0259] It should be noted that the computer device 1000 can execute the filtering method in the embodiments of this application, and can also execute the filtering model training method in the embodiments of this application.

[0260] Those skilled in the art will appreciate that the functionality described in conjunction with the various illustrative logic blocks, modules, and algorithmic steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality described by the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., based on a communication protocol). In this way, the computer-readable medium may substantially correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this application. A computer program product may comprise a computer-readable medium.

[0261] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is properly referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are specifically directed to non-temporary tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, DVDs, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above items should also be included in the scope of computer-readable media.

[0262] Instructions can be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described in the various illustrative logic blocks, modules, and steps described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Moreover, the techniques can be fully implemented in one or more circuit or logic elements. In one example, the various illustrative logic blocks, units, and modules in the encoder and decoder can be understood as corresponding circuit devices or logic elements.

[0263] The techniques described in this application can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The various components, modules, or units described in this application are intended to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with suitable software and / or firmware within a codec hardware unit, or provided via interoperable hardware units (including one or more processors as described above).

[0264] In other words, the above embodiments can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0265] In some embodiments, an encoding end device is provided, the encoding end device including a memory and a processor;

[0266] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the filtering method described above.

[0267] In some embodiments, a decoding end device is provided, the decoding end device including a memory and a processor;

[0268] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the filtering method described above.

[0269] In some embodiments, a filter model training device is provided, the filter model training device including a memory and a processor;

[0270] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the filtering model training method described above.

[0271] In some embodiments, a computer-readable storage medium is provided that stores instructions which, when executed on the computer, cause the computer to perform the steps of the method described above.

[0272] In some embodiments, a computer program is provided that, when executed, implements the methods described above.

[0273] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0274] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the quantization parameters, filtering models, current coding blocks, and reconstruction blocks involved in the embodiments of this application were all obtained with full authorization.

[0275] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A filtering method, characterized in that, Applied to the encoding end, the method includes: Based on the quantization parameters of the target image, K sets of filtering models are determined. Each set of filtering models includes M filtering models. The same set of filtering models corresponds to the same quantization parameter, and different sets of filtering models correspond to different quantization parameters. K and M are both integers greater than 1. Determine the reconstructed block corresponding to the current coded block in the target image; A target filtering model is determined from the K groups of filtering models. The target filtering model is the filtering model that minimizes the coding distortion after filtering the reconstructed block, and the coding distortion of the reconstructed block after filtering it with the target filtering model is less than the coding distortion of the reconstructed block. The reconstructed block is filtered based on the target filtering model.

2. The method as described in claim 1, characterized in that, Determining the target filtering model from the K groups of filtering models includes: Based on the current coding block, the reconstructed block, and the K sets of filtering models, filter indication information corresponding to the reconstructed block is determined. The filter indication information is used to indicate whether the reconstructed block needs to be filtered. When the filtering indication information indicates that the reconstructed block needs filtering, a target filtering model is determined from the K groups of filtering models.

3. The method as described in claim 2, characterized in that, The step of determining the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models includes: The reconstructed block is input into each of the K groups of filtering models to obtain K*M filtering blocks; Based on the current coding block, the reconstructed block, and the K*M filtering blocks, determine the rate-distortion cost corresponding to the reconstructed block and the rate-distortion cost corresponding to each filtering block; If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtered block, then the filtering indication information is determined to be the first indication information, which is used to indicate that the reconstructed block needs to be filtered. If the rate-distortion cost corresponding to the reconstruction block is less than the rate-distortion cost corresponding to each filter block, then the filtering indication information is determined to be the second indication information, which is used to indicate that the reconstruction block does not need to be filtered.

4. The method as described in claim 2 or 3, characterized in that, The method further includes: If the filtering indication information indicates that the reconstructed block does not require filtering, then the reconstructed block is not filtered.

5. The method as described in claim 2 or 3, characterized in that, After determining the filtering indication information corresponding to the reconstructed block based on the current coding block, the reconstructed block, and the K sets of filtering models, the method further includes: The filtering instruction information is encoded into the bitstream.

6. The method according to any one of claims 1-3, characterized in that, After determining the target filtering model from the K groups of filtering models, the process further includes: The target index is encoded into the bitstream, and the target index is used to indicate the target filtering model.

7. The method according to any one of claims 1-3, characterized in that, After determining the K sets of filtering models based on the quantization parameters of the target image, the process further includes: The quantization parameters corresponding to the K groups of filtering models are encoded into the bitstream.

8. A filtering method, characterized in that, Applied to the decoding end, the method includes: K groups of filtering models are determined. Each group of filtering models in the K groups includes M filtering models. The same group of filtering models corresponds to the same quantization parameter, and different groups of filtering models correspond to different quantization parameters. K and M are both integers greater than 1. Determine the reconstructed blocks based on the bitstream; Determine the target filtering model among the K groups of filtering models. The target filtering model is the filtering model that minimizes the encoding distortion after filtering the reconstructed block, and the encoding distortion of the reconstructed block after filtering it through the target filtering model is less than the encoding distortion of the reconstructed block. The reconstructed block is filtered based on the target filtering model.

9. The method as described in claim 8, characterized in that, The determination of the K sets of filtering models includes: The K groups of filtering models are determined based on the quantization parameters of the target image to which the reconstructed block belongs.

10. The method as described in claim 8, characterized in that, The determination of the K sets of filtering models includes: The quantization parameters corresponding to the K groups of filtering models are parsed from the bitstream; Based on the quantization parameters corresponding to the K-group filtering models, the K-group filtering models are determined.

11. The method according to any one of claims 8-10, characterized in that, Determining the target filtering model among the K groups of filtering models includes: Determine the filtering indication information of the reconstructed block, the filtering indication information being used to indicate whether the reconstructed block needs filtering; When the filtering indication information indicates that the reconstruction block needs filtering, the target filtering model in the K groups of filtering models is determined.

12. The method as described in claim 11, characterized in that, The step of determining the filtering indication information for the reconstructed block includes: The filtering indication information is parsed from the bitstream.

13. The method according to any one of claims 8-10, 12, characterized in that, Determining the target filtering model among the K groups of filtering models includes: The target index is parsed from the bitstream, and the target index is used to indicate the target filtering model; The target filtering model is determined based on the target index.

14. A method for training a filtering model, characterized in that, The method includes: Obtain a training sample set, which includes multiple sample coding blocks and a reconstruction block corresponding to each sample coding block, wherein the quantization parameter of the image to which the multiple sample coding blocks belong is the same quantization parameter; Based on the training sample set, the filtering model to be trained is trained to obtain an initial filtering model; The training sample set is divided into M initial sample subsets, each initial sample subset including at least two sample coding blocks and the reconstruction blocks corresponding to the at least two sample coding blocks; Based on the M initial sample subsets, the initial filtering model is trained respectively to obtain M optimized filtering models; Based on the training sample set, the M optimized filtering models are trained to obtain a set of filtering models.

15. The method as described in claim 14, characterized in that, The step of dividing the training sample set into M initial sample subsets includes: The reconstructed blocks corresponding to the plurality of sample coding blocks are input into the initial filtering model to obtain the filtering block corresponding to each sample coding block; Based on the plurality of sample coding blocks and the filter block corresponding to each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined; The multiple sample coding blocks are sorted according to their peak signal-to-noise ratio (PSNR). According to the sorting results, the training sample set is divided into M initial sample subsets, and each initial sample subset includes sample coding blocks that are at least two consecutive sample coding blocks in the sorting results.

16. The method as described in claim 14 or 15, characterized in that, The step of training the M optimized filtering models based on the training sample set to obtain a set of filtering models includes: Based on the training sample set, the M optimized filtering models are trained through iterative iteration; wherein the i-th iteration in the iterative process includes the following steps: Based on the multiple sample coding blocks and the reconstruction block corresponding to each sample coding block, the training sample set is divided into M optimized sample subsets. The M optimized sample subsets correspond one-to-one with the M filtering models processed in the i-th iteration, wherein the M filtering models processed in the first iteration are the M optimized filtering models. Based on the M optimized sample subsets, the M filtering models processed in the i-th iteration are trained; If i is less than the iteration number threshold, then the M filter models processed in the i-th iteration are used as the M filter models processed in the (i+1)-th iteration, and the (i+1)-th iteration is performed. If i is greater than or equal to the iteration number threshold, then the M filter models processed in the i-th iteration after training are determined as a set of filter models.

17. The method as described in claim 16, characterized in that, The training sample set is divided into M optimized sample subsets based on the plurality of sample coding blocks and the reconstruction block corresponding to each sample coding block, including: The reconstructed blocks corresponding to the plurality of sample coding blocks are input into the M filtering models of the i-th iteration to obtain the M filtering blocks corresponding to each sample coding block; Based on the plurality of sample coding blocks and the M filter blocks corresponding to each sample coding block, the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined; Based on the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block, the training sample set is divided into the M optimized sample subsets, wherein each sample coding block is located in the optimized sample subset of the filter model corresponding to the filter block with the largest PSNR among its M filter blocks.

18. A filtering device, characterized in that, Applied to the encoding end, the device includes: The first determining module is used to determine K groups of filtering models based on the quantization parameters of the target image. Each group of filtering models in the K groups includes M filtering models, and the same group of filtering models corresponds to the same quantization parameter. Different groups of filtering models correspond to different quantization parameters. K and M are both integers greater than 1. The second determining module is used to determine the reconstruction block corresponding to the current coding block in the target image; The third determining module is used to determine a target filtering model from the K groups of filtering models. The target filtering model is the filtering model that minimizes the encoding distortion after filtering the reconstructed block, and the encoding distortion of the reconstructed block after filtering it with the target filtering model is less than the encoding distortion of the reconstructed block. The first filtering module is used to filter the reconstructed block based on the target filtering model.

19. The apparatus as claimed in claim 18, characterized in that, The third determining module includes: The first determining unit is configured to determine filtering indication information corresponding to the reconstruction block based on the current coding block, the reconstruction block, and the K sets of filtering models. The filtering indication information is used to indicate whether the reconstruction block needs filtering. The second determining unit is used to determine a target filtering model from the K groups of filtering models when the filtering indication information indicates that the reconstruction block needs filtering.

20. The apparatus as claimed in claim 19, characterized in that, The first determining unit is specifically used for: The reconstructed block is input into each of the K groups of filtering models to obtain K*M filtering blocks; Based on the current coding block, the reconstructed block, and the K*M filtering blocks, determine the rate-distortion cost corresponding to the reconstructed block and the rate-distortion cost corresponding to each filtering block; If the rate-distortion cost corresponding to the reconstructed block is not less than the rate-distortion cost corresponding to each filtered block, then the filtering indication information is determined to be the first indication information, which is used to indicate that the reconstructed block needs to be filtered. If the rate-distortion cost corresponding to the reconstruction block is less than the rate-distortion cost corresponding to each filter block, then the filtering indication information is determined to be the second indication information, which is used to indicate that the reconstruction block does not need to be filtered.

21. The apparatus as claimed in claim 19 or 20, characterized in that, The device further includes: The second filtering module is used to not filter the reconstructed block when the filtering indication information indicates that the reconstructed block does not need to be filtered.

22. The apparatus as claimed in claim 19 or 20, characterized in that, The third determining module further includes: The encoding unit is used to encode the filtering indication information into the bitstream.

23. The apparatus according to any one of claims 18-20, characterized in that, The device further includes: The first encoding module is used to encode the target index into the bitstream, wherein the target index is used to indicate the target filtering model.

24. The apparatus according to any one of claims 18-20, characterized in that, The device further includes: The second encoding module is used to encode the quantization parameters corresponding to the K groups of filtering models into the bitstream.

25. A filtering device, characterized in that, The device, applied at the decoding end, includes: The first determining module is used to determine K groups of filtering models. Each group of filtering models in the K groups includes M filtering models. The same group of filtering models corresponds to the same quantization parameter, and different groups of filtering models correspond to different quantization parameters. K and M are both integers greater than 1. The second determining module is used to determine the reconstruction block based on the bitstream; The third determining module is used to determine the target filtering model among the K groups of filtering models. The target filtering model is the filtering model that minimizes the encoding distortion after filtering the reconstructed block, and the encoding distortion of the reconstructed block after filtering it through the target filtering model is less than the encoding distortion of the reconstructed block. The filtering module is used to filter the reconstructed block based on the target filtering model.

26. The apparatus as claimed in claim 25, characterized in that, The first determining module is specifically used for: The K groups of filtering models are determined based on the quantization parameters of the target image to which the reconstructed block belongs.

27. The apparatus as claimed in claim 25, characterized in that, The first determining module is specifically used for: The quantization parameters corresponding to the K groups of filtering models are parsed from the bitstream; Based on the quantization parameters corresponding to the K-group filtering models, the K-group filtering models are determined.

28. The apparatus according to any one of claims 25-27, characterized in that, The third determining module includes: The first determining unit is used to determine the filtering indication information of the reconstructed block, wherein the filtering indication information is used to indicate whether the reconstructed block needs to be filtered; The second determining unit is used to determine the target filtering model among the K groups of filtering models when the filtering indication information indicates that the reconstruction block needs filtering.

29. The apparatus as claimed in claim 28, characterized in that, The first determining unit is specifically used for: The filtering indication information is parsed from the bitstream.

30. The apparatus according to any one of claims 25-27, 29, characterized in that, The third determining module is specifically used for: The target index is parsed from the bitstream, and the target index is used to indicate the target filtering model; The target filtering model is determined based on the target index.

31. A filter model training device, characterized in that, The device includes: The acquisition module is used to acquire a training sample set, which includes multiple sample coding blocks and a reconstruction block corresponding to each sample coding block. The quantization parameter of the image to which the multiple sample coding blocks belong is the same quantization parameter. The first training module is used to train the filter model to be trained based on the training sample set to obtain an initial filter model. A partitioning module is used to divide the training sample set into M initial sample subsets, each initial sample subset including at least two sample coding blocks and a reconstruction block corresponding to the at least two sample coding blocks; The second training module is used to train the initial filtering model based on the M initial sample subsets to obtain M optimized filtering models. The third training module is used to train the M optimized filtering models based on the training sample set to obtain a set of filtering models.

32. The apparatus as claimed in claim 31, characterized in that, The partitioning module is specifically used for: The reconstructed blocks corresponding to the plurality of sample coding blocks are input into the initial filtering model to obtain the filtering block corresponding to each sample coding block; Based on the plurality of sample coding blocks and the filter block corresponding to each sample coding block, the peak signal-to-noise ratio of the filter block corresponding to each sample coding block is determined; The multiple sample coding blocks are sorted according to their peak signal-to-noise ratio (PSNR). According to the sorting results, the training sample set is divided into M initial sample subsets, and each initial sample subset includes sample coding blocks that are at least two consecutive sample coding blocks in the sorting results.

33. The apparatus as claimed in claim 31 or 32, characterized in that, The third training module is specifically used for: Based on the training sample set, the M optimized filtering models are trained through iterative iteration; wherein the i-th iteration in the iterative process includes the following steps: Based on the multiple sample coding blocks and the reconstruction block corresponding to each sample coding block, the training sample set is divided into M optimized sample subsets. The M optimized sample subsets correspond one-to-one with the M filtering models processed in the i-th iteration, wherein the M filtering models processed in the first iteration are the M optimized filtering models. Based on the M optimized sample subsets, the M filtering models processed in the i-th iteration are trained; If i is less than the iteration number threshold, then the M filter models processed in the i-th iteration are used as the M filter models processed in the (i+1)-th iteration, and the (i+1)-th iteration is performed. If i is greater than or equal to the iteration number threshold, then the M filter models processed in the i-th iteration after training are determined as a set of filter models.

34. The apparatus as claimed in claim 33, characterized in that, The third training module is specifically used for: The reconstructed blocks corresponding to the plurality of sample coding blocks are input into the M filtering models of the i-th iteration to obtain the M filtering blocks corresponding to each sample coding block; Based on the plurality of sample coding blocks and the M filter blocks corresponding to each sample coding block, the peak signal-to-noise ratio of the M filter blocks corresponding to each sample coding block is determined; Based on the peak signal-to-noise ratio (PSNR) of the M filter blocks corresponding to each sample coding block, the training sample set is divided into the M optimized sample subsets, wherein each sample coding block is located in the optimized sample subset of the filter model corresponding to the filter block with the largest PSNR among its M filter blocks.

35. An encoding terminal device, characterized in that, The encoding device includes a memory and a processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the method according to any one of claims 1-7.

36. A decoding device, characterized in that, The decoding device includes a memory and a processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the method according to any one of claims 8-13.

37. A filtering model training device, characterized in that, The filtering model training device includes a memory and a processor, the memory being used to store a computer program, and the processor being configured to execute the computer program stored in the memory to implement the steps of the method according to any one of claims 14-17.

38. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on the computer, cause the computer to perform the steps of the method described in any one of claims 1-17.

39. A computer program product, characterized in that, The computer program product includes instructions that, when executed on the computer, cause the computer to perform the steps of the method according to any one of claims 1-17.

Images