Adaptive parameter set management method and device, equipment and storage medium

Abstract

The invention provides an adaptive parameter set management method and device, equipment and a storage medium, relates to the technical field of video coding, is used for solving the problem of reducing the fullness degree of an ALF APS historical candidate set in a video filtering process, and comprises the following steps: obtaining a target video category and / or a target video feature corresponding to a target video block; and determining a correction factor of each decision factor according to the target video category and / or the target video feature. According to the target correction factor, each decision factor is corrected, a plurality of corrected decision factors are obtained, the probability of filtering indicated by one corrected decision factor is larger than the probability of filtering indicated by the decision factor before correction, and the probability of filtering indicated by the decision factor before correction is larger than the probability of filtering indicated by the decision factor before correction. And the probability that the new APS is used during filtering indicated by one corrected decision factor is greater than the probability that the new APS is used during filtering indicated by the uncorrected decision factor. And storing the obtained at least one new APS to a historical candidate set according to each corrected decision factor.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding technologies, and in particular, to a method, apparatus, device, and storage medium for managing an adaptive parameter set. Background Art

[0002] In the adaptive loop filter (ALF) technology, there are four types of filtering modes: chrominance ALF, luminance ALF, cross-component adaptive loop filter (CCALF) Cb (the difference between the blue component and luminance), and CCALF Cr (the difference between the red component and luminance), and each filtering mode corresponds to an adaptive parameter set (APS) containing filtering parameters.

[0003] Currently, when performing ALF processing on a video, each filtering mode determines a corresponding decision factor according to the current video block in the video, and determines multiple new APSs adopted by the current video block during ALF processing based on multiple decision factors. Then, in the ALF APS historical candidate set, multiple historical APSs adopted by the historical video block are replaced with multiple new APSs adopted by the current video block, providing a filtering reference for subsequent video blocks in the video.

[0004] However, in the above technical solution, the decision factor corresponding to a certain filtering mode in the four filterings may indicate not to adopt a new APS, resulting in a small number of new APSs adopted by the video block. Then, after storing the multiple new APSs adopted by the video block into the ALF APS historical candidate set, the number of APSs corresponding to a certain filtering mode in the ALF APS historical candidate set will be reduced, reducing the fullness of the ALF APS historical candidate set. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method, apparatus, device, and storage medium for managing an adaptive parameter set, which are used to solve the problem of reducing the fullness of the ALF APS historical candidate set during the video filtering process.

[0006] On the one hand, a method for managing an adaptive parameter set is provided. The method includes: an adaptive parameter set management device (hereinafter referred to as the "management device") obtains a target video category and / or target video features corresponding to a target video block. The management device determines a correction factor for each decision factor according to the target video category and / or target video features. One decision factor corresponds to a preset filtering mode. The decision factor is used to indicate whether to filter the video block, and when filtering, it indicates the probability of using any one of a new APS, a historical APS in a historical candidate set, and a preset APS. The management device corrects each decision factor according to the target correction factor to obtain a plurality of corrected decision factors. The probability of a corrected decision factor indicating filtering is greater than the probability of the decision factor before correction indicating filtering, and the probability of a corrected decision factor indicating using a new APS when filtering is greater than the probability of the decision factor before correction indicating using a new APS when filtering. The management device stores at least one obtained new APS into the historical candidate set according to each corrected decision factor.

[0007] On the other hand, a management device for an adaptive parameter set is provided. The device includes: an acquisition module, a processing module, and a storage module.

[0008] The acquisition module is used to obtain a target video category and / or target video features corresponding to a target video block. The processing module is used to determine a correction factor for each decision factor according to the target video category and / or target video features. One decision factor corresponds to a preset filtering mode. The decision factor is used to indicate whether to filter the video block and the probability of using any one of a new APS, a historical APS in a historical candidate set, and a preset APS when filtering. The processing module is further used to correct each decision factor according to the target correction factor to obtain a plurality of corrected decision factors. The probability of a corrected decision factor indicating filtering is greater than the probability of the decision factor before correction indicating filtering, and the probability of a corrected decision factor indicating using a new APS when filtering is greater than the probability of the decision factor before correction indicating using a new APS when filtering. The storage module is used to store at least one obtained new APS into the historical candidate set according to each corrected decision factor.

[0009] On the other hand, a network device is provided, including: a memory and a processor. The memory and the processor are coupled. The memory is used to store a computer program. When the processor executes the computer program, the method for managing an adaptive parameter set according to any one of the above embodiments is implemented.

[0010] On the other hand, a computer-readable storage medium is provided. Computer program instructions are stored on the computer-readable storage medium. When the computer program instructions are executed by a processor, the method for managing an adaptive parameter set according to any one of the above embodiments is implemented.

[0011] In another aspect, a computer program product is provided, which includes computer program instructions that, when executed by a processor, implement the management method of the adaptive parameter set in any of the above embodiments.

[0012] In the embodiments of the present disclosure, when making a filtering decision for different filtering modes, the decision factors in the filtering decision process are corrected according to the video category and video features corresponding to the video block, so that the corrected decision factors are more inclined to perform filtering in the filtering decision process and tend to use the new APS when filtering, ensuring that the number of new APSs for the video block in different filtering modes is large. After all the new APSs of the video block are added to the historical candidate set, it is avoided to reduce the number of APSs corresponding to a certain filtering mode in the historical candidate set, ensuring the fullness of the historical candidate set. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions in the present disclosure, the drawings required for some embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only the drawings of some embodiments of the present disclosure, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0014] Figure 1 A structural block diagram of a video encoder provided for some embodiments of the present disclosure;

[0015] Figure 2 A structural block diagram of a video decoder provided for some embodiments of the present disclosure;

[0016] Figure 3 A schematic flow diagram of loop filtering in the H.266 / VVC standard provided for some embodiments of the present disclosure;

[0017] Figure 4 An example schematic diagram of the internal implementation form of an ALF APS provided for some embodiments of the present disclosure;

[0018] Figure 5 An example schematic diagram of an ALF decision candidate provided for some embodiments of the present disclosure;

[0019] Figure 6 An example schematic diagram of managing APS provided for some embodiments of the present disclosure;

[0020] Figure 7 A schematic diagram of a communication system provided for some embodiments of the present disclosure;

[0021] Figure 8 A schematic flow diagram of a management method of an adaptive parameter set provided for some embodiments of the present disclosure;

[0022] Figure 9 An example schematic diagram of an ALF APS provided by some embodiments of the present disclosure;

[0023] Figure 10 Another example schematic diagram of an ALF APS provided by some embodiments of the present disclosure;

[0024] Figure 11 An example schematic diagram of a historical candidate set of an ALF APS provided by some embodiments of the present disclosure;

[0025] Figure 12 Another example schematic diagram of a management APS provided by some embodiments of the present disclosure;

[0026] Figure 13 Another example schematic diagram of a management APS provided by some embodiments of the present disclosure;

[0027] Figure 14 A schematic structural diagram of a management device for an adaptive parameter set provided by some embodiments of the present disclosure;

[0028] Figure 15 A schematic structural diagram of a management device for an adaptive parameter set provided by some embodiments of the present disclosure. Detailed implementation manners

[0029] Next, the technical solutions in the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the present disclosure. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0030] It should be noted that in the present disclosure, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present disclosure should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0031] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0032] In the description of the present disclosure, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. "And / or" herein is merely a relationship describing associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, both A and B exist simultaneously, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more.

[0033] Before introducing the management method of the adaptive parameter set provided by the embodiments of the present disclosure in detail, the implementation environment and application scenarios of the embodiments of the present disclosure will be introduced first.

[0034] First, the application scenarios of the embodiments of the present disclosure will be introduced.

[0035] All along, improving the video coding compression efficiency is an effective way to improve the quality of video services, and video coding and decoding technologies have been continuously evolving in the past 30 years. The published H.266 / versatile video coding (VVC) standard has coding efficiency that can represent the highest level in the industry. The ALF technology is a new filtering technology added to the H.266 / VVC standard. In the ALF technology, ALF APS is used to transmit ALF filtering coefficients. ALF APS can include filtering coefficients corresponding to 4 filtering modes: luminance ALF APS, chrominance ALF APS, CCALF Cb APS, and CCALF Cr APS. In the current ALF technology, for a slice, the decision-making processes for whether to adopt the new APS for the 4 filtering modes of luminance ALF, chrominance ALF, CCALF Cb, and CCALF Cr are relatively independent, and decision factors corresponding to each filtering mode are used for judgment. Among them, the new generation of video coding standard H.266 / VVC coding framework jointly developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) Joint Video Team includes modules such as intra prediction, inter prediction, transformation, quantization, loop filtering, and entropy coding.

[0036] Figure 1 is a structural block diagram of a video encoder provided by an embodiment, as Figure 1As shown below, the overall framework process of the encoding end is as follows. During encoding, the input video is divided into video blocks for subsequent processing. Here, taking the video block as the coding tree unit (CTU) as an example for description. Of course, it can be understood that when the video block is of other types, the implementation process is similar.

[0037] (1) The input video is first divided into frames for CTU partitioning.

[0038] (2) The partitioned CTUs are sent to the intra / inter-frame prediction module for predictive coding. Among them, the intra-frame prediction module is mainly used to remove the spatial correlation of the image, and predict the current pixel block through the reconstructed CTU information after encoding to remove spatial redundant information; the inter-frame prediction module is mainly used to remove the temporal correlation of the image. By using the encoded image as the reference image of the current frame, the motion information of each CTU is obtained, thereby removing temporal redundancy.

[0039] (3) Then, the obtained intra / inter-frame prediction values are subtracted from the original CTUs to obtain the residual values, and then the residual values are transformed and quantized to remove the frequency-domain correlation and perform lossy compression on the data. Transform coding transforms the image from the spatial domain signal to the frequency domain, concentrating the energy in the low-frequency region. The quantization module can reduce the dynamic range of image coding.

[0040] (4) Then, all the coding parameters and residual values are entropy-coded to form a binary stream for storage or transmission. The output data of the entropy-coding module is the bitstream after compressing the original video. Figure 1 Here, context-based adaptive binary arithmetic coding (CABAC) and header information coding are taken as examples for entropy coding.

[0041] (5) The intra / inter-frame prediction values and the residual values after inverse quantization and inverse transformation are added to obtain the block reconstruction values, and finally the reconstructed image is formed.

[0042] (6) The reconstructed image undergoes a filtering operation through a loop filter and is stored in the image buffer as a reference image for later use.

[0043] H.266 / VVC adopts a CTU-based hybrid coding framework, and distortion effects such as blocking artifacts, ringing artifacts, color deviation, and image blurring still exist in the compressed video using the H.266 / VVC standard. To reduce the impact of such distortions on video quality, in-loop filtering technology is adopted in H.266 / VVC. The in-loop filtering technology in H.266 / VVC includes: luma mapping with chroma scaling (LMCS), deblocking filter (DBF), sample adaptive offset (SAO), and ALF. LMCS improves the compression efficiency by redistributing codewords for information within the dynamic range; DBF is used to reduce blocking artifacts; SAO is used to improve ringing artifacts; and ALF can reduce decoding errors.

[0044] Figure 2 is the structural block diagram of a video decoder provided by an embodiment. As Figure 2 shown, the framework process at the video decoding end is as follows.

[0045] (1) Parse the bitstream to obtain the prediction mode and get the intra / inter prediction values.

[0046] (2) Perform inverse transform and inverse quantization on the residual values obtained by parsing the bitstream.

[0047] Header information decoding and CABAC decoding are required during the parsing process.

[0048] (3) Add the intra / inter prediction values and the residual values after inverse quantization and inverse transform to obtain the CTU reconstruction value, and finally form the reconstructed image.

[0049] (4) The reconstructed image is filtered through the in-loop filter and stored in the image buffer as a reference image for later use.

[0050] Figure 3 is the schematic diagram of the in-loop filtering process in the H.266 / VVC standard. As Figure 3 shown, whether at the encoding end or the decoding end, the connection order of the various filtering modules in the in-loop filtering technology is successively: the LMCS module, the DBF module, the SAO module, and the ALF module, that is, the filtering order is successively: LMCS, DBF, SAO, and ALF.

[0051] Figure 4It is a schematic diagram of an internal implementation form of ALF APS. ALF APS is used to transmit ALF filter coefficients, and can also be called ALF filter parameters. In this disclosure, ALF filter coefficients and ALF filter parameters refer to the same thing. ALF APS can include filter coefficients corresponding to 4 filtering modes: luminance ALF APS, chrominance ALF APS, CCALF Cb APS, and CCALF Cr APS. Inside ALF APS, there can be 4 modules, namely: luminance ALF, chrominance ALF, CCALF Cb, and CCALF Cr. Whether the new APS of each module is used is independent of each other, and each module has a corresponding identifier to indicate whether the new APS is used.

[0052] alf_luma_filter_signal_flag: Indicates whether there are filtering parameters related to luminance ALF inside the current ALF APS. If it is equal to the first preset value, it means there are; if it is equal to the second preset value, it means there are none.

[0053] Exemplarily, the first preset value can be 1, and the second preset value can be 0.

[0054] alf_chroma_filter_signal_flag: Indicates whether there are filtering parameters related to chrominance ALF inside the current ALF APS. If it is equal to the first preset value, it means there are; if it is equal to the second preset value, it means there are none.

[0055] alf_cc_cb_filter_signal_flag: Indicates whether there are filtering parameters related to the CCALF Cb mode inside the current ALF APS. If it is equal to the first preset value, it means there are; if it is equal to the second preset value, it means there are none.

[0056] alf_cc_cr_filter_signal_flag: Indicates whether there are filtering parameters related to the CCALF Cr mode inside the current ALF APS. If it is equal to the first preset value, it means there are; if it is equal to the second preset value, it means there are none.

[0057] It should be noted that whether there are filtering parameters related to each filtering mode inside the current ALF APS refers to whether there are new APS related to each filtering mode inside the current ALF APS. That is, whether to use the new APS related to each filtering mode. The new APS related to each filtering mode is based on the original frame and the reconstructed frame, using the idea of the Wiener filter, with the minimum mean squared error (MSE) between the original frame and the reconstructed frame as the optimization goal, listing the relevant Wiener-Hopf formula for the two, and obtaining the ALF filter coefficients according to the solution results of the equation.

[0058] Whether each filtering mode inside the ALF APS uses the corresponding new APS independently. It is necessary to judge through the rate-distortion criterion when making decisions on each filtering mode of the ALF. If all four identifier decisions are the second preset value, the new APS of the ALF is not used for the current video block. However, once there is one identifier being the first preset value, then the new APS of the ALF for the current video block will be used. Using the new APS of the ALF for the current video block in this embodiment means filtering according to the new APS of the ALF to achieve image reconstruction. Further, the new APS of the ALF can also be transmitted in the bitstream. Simply put, for the ALF APS transmitted to the bitstream, there is at least one set of filtering coefficients for one filtering mode inside it, which can be and any one of these 4 sets of filtering coefficients, and at most there can be 4 sets of filtering coefficients, that is, the above 4 sets of filtering coefficients all exist.

[0059] During the ALF operation, an ALF APS historical candidate set is also maintained, which plays a major role in the acquisition of historical APS. When it is determined that a new APS is used, the ALF APS historical candidate set is also updated according to the first-in, first-out rule. The acquisition of historical APS in the ALF depends on the ALF APS historical candidate set, and the number of historical sets of the current video block can be calculated through the historical candidate set. When acquiring the historical set candidate, it is necessary to judge whether the APS in the historical candidate set can be used by the current video block. The judgment conditions are:

[0060] (1) The time layer for transmitting this APS is lower than or equal to the time layer of the current video block;

[0061] (2) There is a set of filters corresponding to the ALF mode in this APS.

[0062] Only the APS that meets the above two conditions will be selected as the historical APS of the current video block.

[0063] In addition to the rule restrictions on the acquisition of historical APS, the historical candidate set will also have corresponding differences under different coding configurations.

[0064] In the all intra (AI) configuration, all frames are in the all intra mode, so historical APS cannot be used, and thus no historical candidate set will be formed.

[0065] In the random access (RA) configuration, the ALF APS historical candidate set is cleared every time an I frame is passed, which will limit the role of historical APS.

[0066] In the low-delay mode (LD) configuration, since all video blocks belong to the same time layer in the official LD configuration, B / P frames can be unrestricted by the time layer in obtaining the history set. Also, there is no mechanism to clear the history candidate set under LD. Once an APS has been transmitted, it will remain in the ALF APS history candidate set until it is overwritten by a new APS with the same identity (ID) number.

[0067] ALF decision refers to determining the filtering coefficients of ALF among the ALF decision candidates. Figure 5 It is a schematic diagram of an example of ALF decision candidates provided by an embodiment. As Figure 5 shown, the ALF decision candidates include the following four options.

[0068] First, when alf_enable_flag = 0, no ALF is performed (skip ALF).

[0069] When alf_enable_flag = 1, there are the following three options.

[0070] Second, new APS (new ALF APS), where the index of the new APS and the filtering coefficients of the new APS need to be transmitted.

[0071] Third, offline-trained filter sets, only the index of the offline-trained filter sets needs to be transmitted. When it is determined to use the offline-trained filter sets, the index of the offline-trained filter sets also needs to be determined.

[0072] Fourth, historical APS, only the index of the historical APS needs to be transmitted. The historical APS is calculated online from the previous frame.

[0073] The ALF decision process uses the rate-distortion criterion for judgment. A decision factor will be introduced during the judgment process. The decision factor needs to be calculated before encoding the current video block, and parameters such as the quantization parameter (QP) in the configuration are required for the calculation. Due to the different component characteristics of luminance and chrominance, the settings of the fixed functions are slightly different, so the decision factors for different ALF filtering modes are different.

[0074] Since mainstream video compression is lossy compression, the rate-distortion criterion is used as a tool to measure compression performance and is the most widely used technology at the video compression encoding end. Almost all encoding modules need to use the rate-distortion criterion to determine the final encoding mode. The rate-distortion criterion is based on the principle of rate distortion optimization (RDO). Under the constraint of the encoding bit rate, an optimization problem is established by minimizing the distortion, as shown in Equation 1:

[0075]

[0076] where D is the distortion. Since there is no time-domain and frequency-domain transformation in the loop filter section, the distortion D is uniformly the minimum mean square error MSE. R is the bit rate, and X can represent the partition structure, prediction mode, transform coefficient, etc. In the ALF module, when making a Slice-level decision, X is determined within the range of the ALF decision candidate identifiers shown in Figure 5 and includes skipping ALF, adopting a new APS, a fixed filter set, and the historical APS.

[0077] The constrained problem in the above formula can be transformed into an unconstrained problem by introducing a decision factor (such as the Lagrange factor (lambda, λ)), as shown in Equation 2:

[0078]

[0079] where J represents the rate-distortion cost. The rate-distortion cost can accurately describe the comprehensive influence of the number of bits required for the current mode and the distortion after the action of this mode. If we want to judge the advantages and disadvantages of two filter sets, we can compare their rate-distortion costs. A smaller mode cost indicates better compression performance.

[0080] Generally, in video encoding, the value of the Lagrange factor has a fixed functional relationship with the QP of the current Slice. The Lagrange factor needs to be calculated before encoding the current Slice, and the QP and other parameters in the configuration will be used during the calculation. Due to the different characteristics of the luminance and chrominance components, the settings of the above fixed function are slightly different, so the Lagrange factors of different components are different.

[0081] In addition, the input video formats of existing video coding standards (H.265 / high efficiency video coding (HEVC), H.266 / VVC, etc.) are all YUV files. Compared with red green blue (RGB) files, the YUV file format can be compressed once when the source is collected. A pixel in an RGB file consists of three components: R, G, and B, while a pixel in a YUV file consists of three components: Y, U, and V. Since the human visual system is more sensitive to the Y component and not to the U and V components, formats such as 4:2:0, 4:2:2, and 4:4:4, which are Y:U:V data ratios, have emerged.

[0082] In order to more comprehensively test the overall performance of the standard reference software, in the general test sequence of the H.266 / VVC standard, different categories of video sequences have differences in resolution, bit depth, content characteristics, etc., as shown in Table 1, which shows the differences in the characteristics of the general test sequence of the H.266 / VVC standard. Among them, video resolution measures the video pixel width and pixel height, which helps to determine the quality of the video and the clarity or fidelity of its display. Video bit depth is mainly used to express the number of bits used for a single color component of a pixel, also often called color depth or quantization depth, which determines the accuracy of the color that can be expressed by a digital image. Video content characteristics often include texture complexity and motion complexity. Texture usually refers to the local irregular but macroscopic regular characteristics of an image. Texture complexity is a content feature characterization indicator in the spatial domain, including mean, variance, contrast, dissimilarity, correlation, entropy, etc. Motion usually refers to the change of image content between video frames. Motion complexity is a content feature characterization indicator in the temporal domain, including the temporal change of the video sequence (dramatic / slow).

[0083] Table 1 Common test sequence feature differences of H.266 / VVC standard

[0084]

[0085]

[0086] In summary, the ALF technology uses the idea of Wiener filter, takes the MSE between the original frame and the reconstructed frame as the optimization target, lists the Wiener-Hopf formula related to the two, and obtains the final ALF filtering coefficients according to the solution results of the equations. It more directly reduces the MSE and achieves the purpose of improving the peak signal-to-noise ratio of the image. In the ALF technology, the ALF adaptive parameter set (APS) is used to transmit the ALF filtering coefficients. The ALF APS can include the filtering coefficients corresponding to 4 filtering modes: luminance ALF APS, chrominance ALF APS, Cross-Component ALF (CCALF) Cb APS, and CCALF Cr APS.

[0087] In the current ALF technology, for a slice, the decision-making processes of whether to adopt the new APS for the four filtering modes of luminance ALF, chrominance ALF, CCALF Cb, and CCALF Cr are relatively independent, and each filtering mode makes a comparison and judgment based on its own performance. During the judgment process, the decision factors corresponding to each filtering mode are used for judgment. However, when transmitting the APS related to ALF, different filtering modes share the same ALF APS identifier (ID). If for the same slice, the above four APSs are all decided to be transmitted, then the four APSs share one ID, and the benefit of this APS ID can be maximized. On the contrary, if only one APS is transmitted inside an ALF APS ID, then an ALF APS ID only corresponds to one APS, and at the same time, it may also cover several APSs corresponding to the ID before, which may instead reduce the number of APSs corresponding to some ALF modes in the ALF APS historical candidate set. When the ALF APS is updated, if the number of APS types transmitted inside the same ALF APS ID is small, it will lead to a reduction in the historical candidate set during the ALF decision-making process, and the role of ALF cannot be fully exerted.

[0088] Exemplarily, as Figure 6 shown, the encoding-end process of ALF in the existing mechanism is as follows.

[0089] (1) The encoding end calculates the new APS of luminance ALF and the new APS of chrominance ALF for the current slice.

[0090] (2) The encoding end determines whether the luminance ALF decision is enabled.

[0091] If the luminance ALF decision is not enabled, the encoding end does not perform ALF filtering.

[0092] (3) If the luminance ALF decision is enabled, the encoding end makes a luminance ALF decision.

[0093] Perform RDO decision without ALF, new APS, fixed filter set, and historical APS. If the decision result uses the new APS of luma, set alf_luma_filter_signal_flag to 1; otherwise, set it to 0.

[0094] (4) If the luma ALF decision is enabled, the encoder makes a decision on chroma ALF.

[0095] Perform RDO decision without ALF, new APS, and historical APS. If the decision result uses the new APS of chroma, set alf_chroma_filter_signal_flag to 1; otherwise, set it to 0.

[0096] (5) The encoder reconstructs the image after luma ALF and chroma ALF according to the decision results of luma ALF and chroma ALF.

[0097] (6) The encoder calculates the new APS of CCALF Cb and the new APS of CCALF Cr for the current slice.

[0098] (7) If the luma ALF decision is enabled, the encoder makes a decision on CCALF Cb.

[0099] Perform RDO decision without ALF, new APS, and historical APS. If the decision result uses the new APS of CCALF Cb, set alf_cc_cb_filter_signal_flag to 1; otherwise, set it to 0.

[0100] (8) If the luma ALF decision is enabled, the encoder makes a decision on CCALF Cr.

[0101] Perform RDO decision without ALF, new APS, and historical APS. If the decision result uses the new APS of CCALF Cr, set alf_cc_cr_filter_signal_flag to 1; otherwise, set it to 0.

[0102] (9) The encoder reconstructs the image after the action of CCALF according to the decision results of CCALF Cb and CCALF Cr.

[0103] (10) The encoder determines whether at least one of the identifiers of multiple filtering modes is 1.

[0104] Among them, the identifiers of multiple filtering modes include: alf_luma_filter_signal_flag, alf_chroma_filter_signal_flag, alf_cc_cb_filter_signal_flag, alf_cc_cr_filter_signal_flag.

[0105] That is, alf_luma_filter_signal_flag ∨ alf_chroma_filter_signal_flag ∨

[0106] alf_cc_cb_filter_signal_flag ∨ alf_cc_cr_filter_signal_flag == 1.

[0107] If at least one of the above four identifiers is 1, the encoding end executes (11).

[0108] (11) The encoding end transmits the current new APS.

[0109] That is to say, if at least one of the above four identifiers is 1, the encoding end transmits the current new APS into the bitstream.

[0110] It can be understood that in the existing mechanism, there may be a decision factor corresponding to a certain filtering mode in the 4 types of filtering indicating not to use the new APS, resulting in a smaller number of new APSs adopted by the video block. Furthermore, after storing the multiple new APSs adopted by the video block into the ALF APS historical candidate set, the number of APSs corresponding to a certain filtering mode in the ALF APS historical candidate set will be reduced. In this way, the design scheme of the ALF decision factor is not good, which will cause a large amount of explicit transmission of luminance ALF APSs, while there are vacancies in chroma-related ALF APSs (chroma ALF APS, CCALF Cb APS, and CCALF Cr APS), resulting in a low fullness degree of the ALF APS historical candidate set.

[0111] Furthermore, although ALF can effectively improve the subjective and objective quality of encoded and decoded videos, in the ALF decision-making and ALF APS update processes, the existing mechanism does not consider the influence of video characteristics on the ALF decision factor. Therefore, when optimizing the design scheme of the ALF decision factor, fully considering video classification or various video characteristics can more specifically improve the utilization efficiency of ALF APSs.

[0112] To solve the above problems, embodiments of the present disclosure provide a method for managing an adaptive parameter set. The method for managing an adaptive parameter set provided by the embodiments of the present disclosure is applied to various filtering optimization scenarios related to video coding, including but not limited to: video transmission and storage, video conferencing systems, video surveillance systems, etc.

[0113] The present disclosure describes by taking H.266 / VVC as an example, which is applicable to any video coding and decoding scheme including the ALF parameter set. Based on the existing ALF mechanism in VVC, there is a problem of insufficient fullness in the ALF APS, especially for chroma-related components, including chroma APS, CCALF Cb APS, and CCALF Cr APS. Therefore, the present disclosure sets appropriate constraints and triggering conditions to recommend the transmission of luminance APS, chroma APS, CCALF Cb APS, and CCALF Cr APS as much as possible, thereby increasing the fullness of the ALF APS historical candidate sets of different components, increasing the number of the ALF APS historical sets, and enhancing the use effect of ALF. For example, according to the decision result of luminance ALF, such as when it is confirmed that the luminance APS is to be transmitted, the Lagrangian factor in the ALF related to chroma can be adaptively adjusted based on video characteristics to optimize the ALF decision related to chroma.

[0114] That is to say, since there are many vacancies in the APS historical candidate sets in the current different filtering modes (such as luminance ALF, chroma ALF, CCALF Cb, CCALF Cr), and the image optimization effect of the filtering technology during video coding is not fully exerted. Therefore, the embodiments of the present disclosure propose to adaptively correct the Lagrangian factors in different filtering modes based on the ALF decision result and / or the fullness of the relevant APS historical candidate sets according to the video content and / or video characteristics, so that the corrected Lagrangian factor is more inclined to perform filtering during the RDO decision process and is inclined to use the new APS during filtering, ensuring that the number of new APSs in the video block is large in different filtering modes, and after all the new APSs of the video block are added to the ALF APS historical candidate set, avoiding reducing the number of APSs corresponding to a certain filtering mode in the ALF APS historical candidate set, so as to improve the fullness of the ALF APS historical candidate set and achieve the purpose of maximizing the filtering technology optimization effect, thereby being able to further improve the coding efficiency and image quality.

[0115] It should be noted that the present disclosure is not limited to the H.266 / VVC standard.

[0116] Next, the implementation environment of the embodiments of the present disclosure will be introduced.

[0117] As Figure 7As shown in the figure, it is a schematic diagram of a communication system provided by an embodiment of the present disclosure. The communication system may include: a management device 701 and a collection device 702. Among them, the management device 701 may communicate with the collection device 702 in a wired / wireless manner.

[0118] Among them, the collection device 702 may collect video and send the collected video to the management device 701.

[0119] The management device 701 may obtain a video block in the video from the collection device 702, and determine the video category and video features corresponding to the video block. Then, the management device 701 may correct the decision factor for each filtering mode so that the corrected decision factor is more inclined to perform filtering and is inclined to use the new APS when filtering. After that, the management device 701 may store the obtained at least one new APS in the ALF APS historical candidate set according to each corrected decision factor.

[0120] It should be noted that the embodiments of the present disclosure do not limit the collection device 702. For example, the collection device 702 may be a driving recorder. For another example, the collection device 702 may be a mobile phone with a video shooting function. For another example, the collection device 702 may be a surveillance camera.

[0121] In the embodiments of the present disclosure, the management device 701 may be a terminal, or the management device 701 may be a server.

[0122] Among them, the terminal may be a device such as a mobile phone, a tablet computer, a desktop type, a laptop, a handheld computer, a notebook computer, an Ultra-mobile Personal Computer (UMPC), a netbook, etc. with a transceiver function. The embodiments of the present disclosure do not impose special restrictions on the specific form of the terminal. It may perform human-computer interaction with the user through one or more of a keyboard, a touchpad, a touch screen, a remote control, voice interaction, or a handwriting device.

[0123] The server may be a single physical server, or may also be a server cluster composed of multiple servers. Or, the server cluster may also be a distributed cluster. Or, the server may be a cloud server. The embodiments of the present disclosure do not limit the specific implementation manner of the server.

[0124] After introducing the application scenarios and implementation environments of the embodiments of the present disclosure, the management method of the adaptive parameter set provided by the embodiments of the present disclosure will be introduced in detail below in combination with the above implementation environment.

[0125] The embodiments of the present disclosure provide a management method for an adaptive parameter set, as Figure 8As shown, the method for managing the adaptive parameter set may include: S801 - S804.

[0126] S801. The management device obtains the target video category and / or target video features corresponding to the target video block.

[0127] As a possible implementation, the target video category is the video category of the target video block, and the target video features are the video features of the target video block.

[0128] That is to say, when the present disclosure filters each video block, it will determine the video category and video features of the filtered video block, thereby providing valuable reference for subsequent correction of the decision factors of different filtering modes and improving the accuracy of the correction results.

[0129] Optionally, the target video block may be a video block in the video to be processed. And the target video category is the category of the video to be processed, and the target video features are the video features of the video to be processed.

[0130] That is to say, when the present disclosure filters all video blocks in a video, it only needs to determine the video category and video features of the entire video, and use the video category and video features of the entire video as the video category and video features corresponding to each video block in the video. In this way, the number of times of determining the video category and video features can be reduced, and the correction efficiency of subsequent decision factors can be improved.

[0131] It should be noted that the present disclosure embodiment does not limit the type of the target video block. The target video block may be any of the following types: sub - picture, tile, slice, coding tree units (CTU), coding units (CU).

[0132] That is to say, the present disclosure embodiment can be applied to different types of filtering objects, expanding the scope of subsequent correction of decision factors.

[0133] S802. The management device determines the correction factor of each decision factor according to the target video category and / or target video features.

[0134] Wherein, one decision factor corresponds to one preset filtering mode. The decision factor is used to indicate whether the video block is filtered, and when filtering, it indicates to use any one of the new APS, the historical APS of the historical candidate set, and the preset APS (i.e., the fixed filtering set).

[0135] In the embodiment of the present disclosure, the preset filtering mode is any one of the following ALF: chroma ALF, luma ALF, CCALF Cb, CCALF Cr.

[0136] That is to say, the embodiments of the present disclosure are applicable to the ALF process of videos, and correct the decision factors of different ALF modes when performing ALF on videos, so that the corrected decision factors can provide richer APS for subsequent ALF, and improve the accuracy of video filtering results.

[0137] As a possible implementation manner, the decision factor can be λ in the RDO decision process.

[0138] That is to say, when filtering a video block, the management device can determine whether to perform filtering on the video block through RDO decision, and which APS to use when performing filtering on the video block. In this way, the video filtering efficiency can be improved, and the accuracy of video filtering results can be improved.

[0139] As a possible implementation manner, the management device stores multiple preset correction factors, multiple preset video categories, and / or multiple preset video features. One preset video category corresponds to one preset correction factor, and one preset video feature corresponds to one preset correction factor. The management device can determine the preset video category that is the same as the target video category from multiple preset video categories according to the target video category, and use the preset correction factor corresponding to the preset video category that is the same as the target video category in the multiple preset video categories as the correction factor of each decision factor to determine the correction factor of each decision factor.

[0140] Optionally, the management device can determine the preset video feature that is the same as the target video feature from multiple preset video features according to the target video feature, and use the preset correction factor corresponding to the preset video feature that is the same as the target video feature in the multiple preset video features as the correction factor of each decision factor to determine the correction factor of each decision factor.

[0141] As another possible implementation manner, the management device stores multiple preset correction factors, multiple preset video weights, multiple preset video categories, multiple preset video features, multiple preset category weights, and multiple preset feature weights. One preset video weight corresponds to one preset correction factor, one preset category weight corresponds to one preset video category, and one preset feature weight corresponds to one preset video feature. The management device can determine the preset video category that is the same as the target video category from multiple preset video categories according to the target video category, and use the preset category weight corresponding to the preset video category that is the same as the target video category in the multiple preset video categories as the target category weight corresponding to the target video category.

[0142] Similarly, the management device can determine, according to the target video feature, the preset video feature identical to the target video feature from multiple preset video features, and use the preset feature weight corresponding to the preset video feature identical to the target video feature among the multiple preset video features as the target feature weight corresponding to the target video category.

[0143] Next, the management device determines the target video weight of the target video block by summing the target category weight and the target feature weight. After that, the management device can determine, according to the target video weight, the preset video weight identical to the target video weight from multiple preset video weights, and use the preset correction factor corresponding to the preset video weight identical to the target video weight among the multiple preset video weights as the correction factor for each decision factor to determine the correction factor for each decision factor.

[0144] S803. The management device corrects each decision factor according to the target correction factor to obtain multiple corrected decision factors.

[0145] Among them, the probability indicated by one corrected decision factor for performing filtering is greater than the probability indicated by the decision factor before correction, and the probability indicated by one corrected decision factor for using a new APS during filtering is greater than the probability indicated by the decision factor before correction for using a new APS during filtering.

[0146] As a possible implementation, the management device can determine the corrected decision factor by calculating the product between the decision factor and the target correction factor.

[0147] The following introduces the influence of the decision factors before and after correction on the APS used in the filtering mode in combination with specific examples.

[0148] Exemplarily, taking the filtering mode as the ALF mode as an example, Figure 9 is a schematic diagram of the ALF APS provided by an embodiment, which shows the APS of different ALF modes determined by the decision factor before correction. Among them, "tick" indicates using a new APS, and "cross" indicates not using a new APS, that is, only the chrominance ALF APS uses a new APS, and the other 3 filtering modes do not use a new APS. Figure 10 is a schematic diagram of the ALF APS provided by another embodiment, which shows the APS of different ALF modes determined by the corrected decision factor for the target video block. Among them, all 4 filtering modes use a new APS.

[0149] That is to say, compared with Figure 9 Figure 10 the obtained ALF APS adds 3 new APSs. These three new APSs can be updated to the historical filtering parameters corresponding to the filtering mode subsequently, thereby improving the fullness of the historical candidate set.

[0150] S804. The management device stores at least one obtained new APS into the historical candidate set according to each corrected decision factor.

[0151] As a possible implementation manner, the management device may, according to each corrected decision factor, determine, through RDO decision-making, multiple first filtering modes executed by a target video block among multiple preset filtering modes, and determine, according to each corrected decision factor, at least one second filtering mode using a new APS from the multiple first filtering modes, and obtain the new APS used by each second filtering mode to obtain at least one new APS. Then, the management device may store at least one obtained new APS into the historical candidate set.

[0152] As a possible implementation manner, the historical candidate set may be an ALF APS historical candidate set, and the historical candidate set may include multiple historical APSs and multiple preset identifiers. One preset identifier corresponds to at least one historical APS, and different preset identifiers are sorted in chronological order. During the process that the management device stores at least one obtained new APS into the historical candidate set, the management device may delete all historical APSs corresponding to a first identifier in the historical candidate set. The first identifier is an identifier among the multiple preset identifiers, and the time corresponding to the first identifier is earlier than the time corresponding to any other identifier among the multiple preset identifiers except the first identifier. Then, the management device may store at least one new APS into the historical candidate set, and update the corresponding relationship between the preset identifier and the APS in the historical candidate set according to the time when at least one new APS is stored into the historical candidate set, so that the time corresponding to a second identifier corresponding to at least one new APS among the multiple preset identifiers is later than the time corresponding to any other identifier among the multiple preset identifiers except the second identifier.

[0153] Exemplarily, Figure 11 is a schematic diagram of an example of the ALF APS historical candidate set provided by an embodiment. As Figure 11 shown, during the ALF operation process, an ALF APS historical candidate set is also maintained, which plays a major role in the acquisition of historical APSs. When it is determined that a new APS is used, the ALF APS historical candidate set is also updated according to the first-in, first-out rule. The acquisition of historical APSs in the ALF depends on the ALF APS historical candidate set. It should be noted that the historical APS may also be referred to as historical filtering parameters.

[0154] Both the encoding end and the decoding end construct the ALF APS historical candidate set in the same way. At most N APSs can be maintained inside the bitstream, and the ID (which can also be called Index) numbers range from 0 to N - 1. Each APS includes historical APSs of 4 filtering modes: and Figure 10 In this case, N is taken as 8 for illustration. Whenever a new APS is used, the old APS with the same ID number is overwritten. The ID number of the new APS corresponds to the target video block, that is, the ID number of the new APS can be determined according to the target video block.

[0155] That is to say, the management device adds multiple new APSs used by the current video block in the video to the historical candidate set, provides a valuable reference for the filtering of subsequent video blocks in the video, and improves the accuracy of the filtering result of the entire video.

[0156] It can be understood that when making a filtering decision for different filtering modes, the decision factors in the filtering decision process are corrected according to the video category and video features corresponding to the video block, so that the corrected decision factors are more inclined to perform filtering during the filtering decision process, and are inclined to use new APSs during filtering, ensuring that the number of new APSs for the video block in different filtering modes is large, and after all the new APSs of the video block are added to the historical candidate set, it is avoided to reduce the number of APSs corresponding to a certain filtering mode in the historical candidate set, ensuring the fullness of the historical candidate set.

[0157] In some embodiments, during the process of the management device obtaining the target video category corresponding to the target video block, the management device can obtain the target video content of the target video block and determine at least one content feature of the target video content, where the content feature is an optical flow feature, a frequency domain feature, a spatial feature, or a depth feature. Then, the management device can classify the target video block according to the at least one content feature to determine the target video category.

[0158] It should be noted that the main goal of video classification is to understand the content contained in the video and determine one or more key themes corresponding to the video. Video classification can use video classification algorithms to automatically analyze the semantic information contained in the video or video segment, automatically label, classify, and describe the video, achieving an accuracy comparable to that of manual classification.

[0159] Video classification usually considers optical flow features, frequency domain features, spatial features, depth features, etc. of video content. Among them, optical flow features can reflect the motion of objects in the video. The optical flow field can be obtained by calculating the motion vectors of pixels between adjacent frames in the video to extract optical flow features. Frequency domain features can reflect the periodic changes in the video, such as periodic motion in the video. The frequency domain features can be obtained by performing a Fourier transform on the video. Spatial features can reflect the appearance and shape of objects in the video. The spatial features can be extracted by processing each frame in the video, such as color, texture, shape, etc. Depth features can reflect the semantic information of objects in the video, such as the category and attributes of objects. By comprehensively using the above features, the video content can be described more comprehensively, thereby improving the accuracy and robustness of video classification.

[0160] It can be understood that by comparing and smoothing the current pixel with surrounding pixels, ALF can reduce the redundant information in the video, remove the noise and interference in the image or video, so as to achieve the purpose of enhancing the video image quality. Based on video classification, the loop filtering technology can be associated with the main features and motion information of the video, so as to implement a more targeted or personalized filtering scheme for different categories of videos and improve the video coding and decoding performance. Preferably, the Lagrangian factor in ALF can be adaptively corrected based on video classification, that is, the optimal Lagrangian factor correction parameter corresponding to each video category can be tested and determined in advance, and then the corresponding optimal Lagrangian factor correction parameter can be selected according to the category to which the current video or video block (such as sub-image Sub-Picture, tile, slice, coding unit CTU or CU) belongs.

[0161] In some embodiments, the target video features may include at least one of the following features: image resolution, texture complexity, motion complexity, color histogram.

[0162] It should be noted that the adjustment of the Lagrangian factor based on video features means analyzing the video features to extract useful information in the video and constructing a fitting relationship between it and the optimal value of the Lagrangian factor (corresponding to the best video quality). Thus, during actual video coding, the optimal Lagrangian factor correction parameter corresponding to the coding object can be obtained only by extracting the video features of the current video or video block (such as sub-image Sub-Picture, tile, slice, coding unit CTU or CU). In obtaining video features, an appropriate feature extraction method can be selected according to the specific application scenario and requirements. Common video features include image resolution, texture complexity, motion complexity (such as optical flow), color histogram, etc.

[0163] It can be understood that by determining different types of video features, valuable references can be provided for the correction of decision factors, improving the accuracy of the corrected decision factors.

[0164] The following introduces the management method of the adaptive parameter set provided by the embodiments of the present disclosure with specific examples.

[0165] Exemplarily, taking the correction of decision factors based on video categories as an example, as Figure 12 shown, the management method of the adaptive parameter set includes:

[0166] Step 1: The management device calculates the new APS of the luminance ALF and the chrominance ALF.

[0167] Among them, according to the principle of Wiener filtering, the management device calculates the new APS of the luminance ALF for the current Slice and the new APS of the chrominance ALF

[0168] Step 2: The management device adjusts λ within the luminance ALF.

[0169] Step 2-a: The management device determines whether the λ correction condition of the luminance ALF is satisfied.

[0170] It should be noted that in this embodiment, preferably, a Lagrange factor correction condition is introduced, that is, when the correction condition is satisfied, the Lagrange factor is adjusted based on the video category, and the present disclosure does not limit the use of any correction condition.

[0171] If the λ correction condition of the luminance ALF is satisfied, the management device executes Step 2-b.

[0172] If the λ correction condition of the luminance ALF is not satisfied, the management device executes Step 2-d.

[0173] Step 2-b: The management device determines the correction parameter k of the Lagrange factor based on the category to which the video or video block belongs (i.e., the video category).

[0174] Step 2-c: The management device adjusts λ of the luminance ALF using k.

[0175] As shown in Formula 3:

[0176]

[0177] Step 2-d: The management device makes a decision on the luminance ALF.

[0178] Perform RDO decision-making among the following candidate modes:

[0179] (1) Do not perform ALF;

[0180] (2) New APS;

[0181] (3) Fixed filter set (i.e., preset APS);

[0182] (4) Historical APS;

[0183] If the decision result is "use the new APS of brightness ALF", then set alf_luma_filter_signal_flag to 1, otherwise set it to 0.

[0184] Step 3: The management device determines whether the brightness ALF decision is enabled.

[0185] If the brightness ALF decision is not enabled, the management device does not perform ALF filtering.

[0186] If the brightness ALF decision is enabled, the management device executes Step 4.

[0187] Step 4: The management device adjusts λ within the chroma ALF.

[0188] Step 4-a: The management device determines whether the λ correction condition of the chroma ALF is satisfied.

[0189] If the λ correction condition of the chroma ALF is satisfied, the management device executes Step 4-b.

[0190] If the λ correction condition of the chroma ALF is not satisfied, the management device executes Step 4-d.

[0191] Step 4-b: The management device determines the correction parameter k of the Lagrangian factor based on the category to which the video or video block belongs (i.e., the video category).

[0192] Step 4-c: The management device adjusts λ of the chroma ALF using k.

[0193] As shown in Equation 4:

[0194]

[0195] Step 4-d: The management device makes a chroma ALF decision.

[0196] Perform RDO decision among the following candidate modes:

[0197] (1) Do not perform ALF;

[0198] (2) New APS;

[0199] (3) Fixed filter set;

[0200] (4) Historical APS;

[0201] If the decision result is "New APS using chroma ALF", then set alf_luma_filter_signal_flag to 1; otherwise, set it to 0.

[0202] Step 5: The management device performs luma ALF and chroma ALF reconstruction.

[0203] The management device reconstructs the image after the luma and chroma ALF operations according to the decision results of the luma and chroma ALF.

[0204] Step 6: The management device calculates the new APS of CCALF Cb and CCALF Cr.

[0205] Among them, the management device calculates the new APS of CCALF Cb for the current Slice according to the principle of Wiener filtering and the new APS of CCALF Cr

[0206] Step 7: The management device adjusts the λ in CCALF Cb.

[0207] Step 7-a: The management device determines whether the λ correction condition for CCALF Cb is met.

[0208] If the λ correction condition for CCALF Cb is met, the management device executes Step 7-b.

[0209] If the λ correction condition for CCALF Cb is not met, the management device executes Step 7-d.

[0210] Step 7-b: The management device determines the correction parameter k of the Lagrange factor based on the category to which the video or video block belongs (i.e., the video category).

[0211] Step 7-c: The management device adjusts the λ of CCALF Cb using k.

[0212] As shown in Formula Five:

[0213]

[0214] Step 7-d: The management device makes a decision on CCALF Cb.

[0215] Perform RDO decision among the following candidate modes:

[0216] (1) Do not perform ALF;

[0217] (2) New APS;

[0218] (3) Fixed filter set;

[0219] (4) Historical APS;

[0220] If the decision result is "new APS using CCALF Cb", then alf_cc_cb

[0221] _filter_signal_flag shall be set to 1, otherwise set to 0.

[0222] Step 8: The management device adjusts λ in CCALF Cr.

[0223] Step 8-a: The management device determines whether the λ correction condition for CCALF Cr is satisfied.

[0224] If the λ correction condition for CCALF Cr is satisfied, the management device executes Step 8-b.

[0225] If the λ correction condition for CCALF Cr is not satisfied, the management device executes Step 8-d.

[0226] Step 8-b: The management device determines the correction parameter k of the Lagrangian factor based on the category to which the video or video block belongs (i.e., the video category).

[0227] Step 8-c: The management device adjusts λ of CCALF Cr using k.

[0228] As shown in Equation (6):

[0229]

[0230] Step 8-d: The management device makes a CCALF Cr decision.

[0231] Perform RDO decision among the following candidate modes:

[0232] (1) Do not perform ALF;

[0233] (2) New APS;

[0234] (3) Fixed filter set;

[0235] (4) Historical APS;

[0236] If the decision result is "new APS using CCALF Cr", then alf_cc_cr

[0237] _filter_signal_flag shall be set to 1, otherwise set to 0.

[0238] Step 9: The management device performs CCALF Cb and CCALF Cr reconstruction.

[0239] The management device reconstructs the image after the actions of CCALF Cb and CCALF Cr according to the decision results of CCALF Cb and CCALF Cr.

[0240] Step 10: The management device determines whether at least one of the identifiers of multiple filtering modes is 1.

[0241] That is, alf_luma_filter_signal_flag ∨ alf_chroma_filter_signal_flag ∨

[0242] alf_cc_cb_filter_signal_flag ∨ alf_cc_cr_filter_signal_flag == 1.

[0243] If at least one of the above four identifiers is 1, the management device executes Step 11.

[0244] If all of the above four identifiers are 0, the management device does not need to execute Step 11.

[0245] Step 11: The management device transmits the current APS.

[0246] The management device transmits the APS of the current transmission target filtering unit (i.e., the target video block) to the bitstream.

[0247] If there is already an APS with the same index number, directly use the new APS to overwrite the old APS.

[0248] Exemplarily, taking the correction of the decision factor based on video features as an example, such as Figure 13 The management method of the adaptive parameter set shown includes:

[0249] Step 1: The management device calculates the new APS of the luma ALF and the chroma ALF.

[0250] Among them, the management device calculates the new APS of the luma ALF for the current Slice respectively according to the principle of Wiener filtering and the new APS of the chroma ALF

[0251] Step 2: The management device adjusts λ in the luma ALF.

[0252] Step 2-a: The management device determines whether the λ correction condition of the luma ALF is satisfied.

[0253] It should be noted that in this embodiment, preferably, the Lagrangian factor correction condition is introduced, that is, when the correction condition is satisfied, the Lagrangian factor is adjusted based on video features, and the present disclosure does not limit the adoption of any correction condition.

[0254] If the λ correction condition for the luminance ALF is satisfied, the management device executes step 2-b.

[0255] If the λ correction condition for the luminance ALF is not satisfied, the management device executes step 2-d.

[0256] Step 2-b: The management device determines the correction parameter k of the Lagrangian factor based on the video or video block feature (i.e., video feature).

[0257] Step 2-c: The management device adjusts the λ of the luminance ALF using k.

[0258] As shown in Equation 3:

[0259]

[0260] Step 2-d: The management device makes a luminance ALF decision.

[0261] Perform RDO decision among the following candidate modes:

[0262] (1) Do not perform ALF;

[0263] (2) New APS;

[0264] (3) Fixed filter set (i.e., preset APS);

[0265] (4) Historical APS;

[0266] If the decision result is "use the new APS of the luminance ALF", the alf_luma_filter_signal_flag needs to be set to 1, otherwise set to 0.

[0267] Step 3: The management device determines whether the luminance ALF decision is enabled.

[0268] If the luminance ALF decision is not enabled, the management device does not perform ALF filtering.

[0269] If the luminance ALF decision is enabled, the management device executes step 4.

[0270] Step 4: The management device adjusts the λ within the chrominance ALF.

[0271] Step 4-a: The management device determines whether the λ correction condition for the chrominance ALF is satisfied.

[0272] If the λ correction condition for the chrominance ALF is satisfied, the management device executes step 4-b.

[0273] If the λ correction condition for the chrominance ALF is not satisfied, the management device executes step 4-d.

[0274] Step 4-b: The management device determines a correction parameter k of the Lagrange factor based on video or video block features (i.e., video features).

[0275] Step 4-c: The management device adjusts λ of the chroma ALF using k.

[0276] As shown in Formula 4:

[0277]

[0278] Step 4-d: The management device makes a chroma ALF decision.

[0279] Perform RDO decision-making among the following candidate modes:

[0280] (1) Do not perform ALF;

[0281] (2) New APS;

[0282] (3) Fixed filter set;

[0283] (4) Historical APS;

[0284] If the decision result is "new APS using chroma ALF", then set alf_luma_filter_signal_flag to 1, otherwise set it to 0.

[0285] Step 5: The management device performs luminance ALF and chroma ALF reconstruction.

[0286] The management device reconstructs the image after the actions of luminance and chroma ALF according to the decision results of luminance and chroma ALF.

[0287] Step 6: The management device calculates the new APS of CCALF Cb and CCALF Cr.

[0288] Among them, the management device calculates the new APS of CCALF Cb for the current Slice respectively according to the principle of Wiener filtering and the new APS of CCALF Cr

[0289] Step 7: The management device adjusts λ within CCALF Cb.

[0290] Step 7-a: The management device determines whether the λ correction condition of CCALF Cb is satisfied.

[0291] If the λ correction condition of CCALF Cb is satisfied, the management device executes Step 7-b.

[0292] If the λ correction condition of CCALF Cb is not satisfied, the management device executes Step 7-d.

[0293] Step 7-b: The management device determines a correction parameter k of the Lagrange factor based on video or video block features (i.e., video features).

[0294] Step 7-c: The management device adjusts λ of CCALF Cb using k.

[0295] As shown in Formula Five:

[0296]

[0297] Step 7-d: The management device makes a CCALF Cb decision.

[0298] Perform RDO decision-making among the following candidate modes:

[0299] (1) Do not perform ALF;

[0300] (2) New APS;

[0301] (3) Fixed filter set;

[0302] (4) Historical APS;

[0303] If the decision result is "use the new APS of CCALF Cb", then alf_cc_cb

[0304] _filter_signal_flag is set to 1, otherwise it is set to 0.

[0305] Step 8: The management device adjusts λ in CCALF Cr.

[0306] Step 8-a: The management device determines whether the λ correction condition of CCALF Cr is satisfied.

[0307] If the λ correction condition of CCALF Cr is satisfied, the management device executes Step 8-b.

[0308] If the λ correction condition of CCALF Cr is not satisfied, the management device executes Step 8-d.

[0309] Step 8-b: The management device determines a correction parameter k of the Lagrange factor based on video or video block features (i.e., video features).

[0310] Step 8-c: The management device adjusts λ of CCALF Cr using k.

[0311] As shown in Formula Six:

[0312]

[0313] Step 8-d: The management device makes a CCALF Cr decision.

[0314] Perform an RDO decision among the following candidate modes:

[0315] (1) Do not perform ALF;

[0316] (2) New APS;

[0317] (3) Fixed filter set;

[0318] (4) Historical APS;

[0319] If the decision result is "new APS using CCALF Cr", then alf_cc_cr

[0320] _filter_signal_flag shall be set to 1, otherwise set to 0.

[0321] Step 9: The management device performs CCALF Cb and CCALF Cr reconstruction.

[0322] The management device reconstructs the image after the actions of CCALF Cb and CCALF Cr according to the decision results of CCALF Cb and CCALF Cr.

[0323] Step 10: The management device determines whether at least one of the identifiers of multiple filtering modes is 1.

[0324] That is, alf_luma_filter_signal_flag ∨ alf_chroma_filter_signal_flag ∨

[0325] alf_cc_cb_filter_signal_flag ∨ alf_cc_cr_filter_signal_flag == 1.

[0326] If at least one of the above four identifiers is 1, the management device executes Step 11.

[0327] If all of the above four identifiers are 0, the management device does not need to execute Step 11.

[0328] Step 11: The management device transmits the current APS.

[0329] The management device transmits the APS of the current transmission target filtering unit (i.e., the target video block) to the bitstream.

[0330] If there already exists an APS with the same index number (index), directly use the new APS to overwrite the old APS.

[0331] It can be understood that, in order to implement the above functions, the management device of the adaptive parameter set includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments of the present disclosure, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0332] The embodiments of the present disclosure can perform function module division on the management device of the adaptive parameter set according to the above method embodiments. For example, each function module can be divided corresponding to each function, or two or more functions can be integrated into one function module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiments of the present disclosure is illustrative, only a logical function division, and there can be other division methods in actual implementation. The following will take the example of dividing each function module corresponding to each function for illustration.

[0333] Figure 14 It is a schematic structural diagram of a management device of an adaptive parameter set provided by an embodiment of the present disclosure. The management device 1400 of the adaptive parameter set can execute the Figure 8 management method of the adaptive parameter set shown above. As Figure 14 shown, the management device 1400 of the adaptive parameter set includes: an acquisition module 1401, a processing module 1402, and a storage module 1403.

[0334] The acquisition module 1401 is used to acquire the target video category and / or target video features corresponding to the target video block. The processing module 1402 is used to determine the correction factor of each decision factor according to the target video category and / or target video features. One decision factor corresponds to one preset filtering mode. The decision factor is used to indicate whether the video block is filtered, and when filtering, it indicates to use any one of the new APS, the historical APS of the historical candidate set, and the preset APS. The processing module 1402 is further used to correct each decision factor according to the target correction factor to obtain a plurality of corrected decision factors. The probability that a corrected decision factor indicates filtering is greater than the probability that the decision factor before correction indicates filtering, and the probability that a corrected decision factor indicates using the new APS when filtering is greater than the probability that the decision factor before correction indicates using the new APS when filtering. The storage module 1403 is used to store at least one obtained new APS into the historical candidate set according to each corrected decision factor.

[0335] Optionally, the historical candidate set includes multiple historical APSs and multiple preset identifiers. One preset identifier corresponds to at least one historical APS, and different preset identifiers are sorted in chronological order. The processing module 1402 is specifically configured to delete all historical APSs corresponding to the first identifier in the historical candidate set. The first identifier is an identifier among the multiple preset identifiers, and the time corresponding to the first identifier is earlier than the time corresponding to any identifier other than the first identifier among the multiple preset identifiers. The storage module 1403 is further configured to store at least one new APS into the historical candidate set. The processing module 1402 is further configured to update the correspondence between the preset identifier and the APS in the historical candidate set according to the time when at least one new APS is stored into the historical candidate set, so that the time corresponding to the second identifier corresponding to at least one new APS among the multiple preset identifiers is later than the time corresponding to any identifier other than the second identifier among the multiple preset identifiers.

[0336] Optionally, the obtaining module 1401 is specifically configured to obtain the target video content of the target video block. The processing module 1402 is further configured to determine at least one content feature of the target video content. The content feature is an optical flow feature, a frequency domain feature, a spatial feature, or a depth feature. The processing module 1402 is further configured to determine the target video category according to at least one content feature.

[0337] Optionally, the target video feature includes at least one of the following features: image resolution, texture complexity, motion complexity, color histogram.

[0338] Optionally, the target video block is a video block in the video to be processed. The target video category is the video category of the video to be processed, and the target video feature is the video feature of the video to be processed. Or, the target video category is the video category of the target video block, and the target video feature is the video feature of the target video block.

[0339] Optionally, the decision factor is the Lagrange factor in the rate-distortion optimization (RDO) decision process.

[0340] Optionally, the preset filtering mode is any one of the following adaptive loop filters (ALFs): chrominance ALF, luminance ALF, component-wise adaptive loop filter (CCALF) difference between the blue component and luminance Cb, CCALF difference between the red component and luminance Cr.

[0341] Optionally, the target video block is any one of the following types: sub-picture, tile, slice, coding tree unit (CTU), coding unit (CU).

[0342] In the case where the functions of the above integrated modules are implemented in the form of hardware, the embodiments of the present disclosure provide another possible network device structure of the management device for the adaptive parameter set involved in the above embodiments. AsFigure 15 As shown, the management device 1500 of the adaptive parameter set includes: a processor 1502 and a bus 1504. Optionally, the management device 1500 of the adaptive parameter set may further include a memory 1501; optionally, the management device 1500 of the adaptive parameter set may further include a communication interface 1503.

[0343] The processor 1502 may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the embodiments of the present disclosure. The processor 1502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the embodiments of the present disclosure. The processor 1502 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0344] The communication interface 1503 is used to connect to other devices through a communication network. The communication network may be an Ethernet, a radio access network, a wireless local area network (WLAN), etc.

[0345] The memory 1501 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium, or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0346] As a possible implementation, the memory 1501 may exist independently of the processor 1502. The memory 1501 may be connected to the processor 1502 through the bus 1504 for storing instructions or program code. When the processor 1502 calls and executes the instructions or program code stored in the memory 1501, the management method of the adaptive parameter set provided by the embodiments of the present disclosure can be implemented.

[0347] In another possible implementation, the memory 1501 may also be integrated with the processor 1502.

[0348] The bus 1504 can be an extended industry standard architecture (EISA) bus or the like. The bus 1504 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 15 only a thick line is used to represent it in Figure 15 , but it does not mean that there is only one bus or one type of bus.

[0349] Some embodiments of the present disclosure provide a computer-readable storage medium (for example, a non-transitory computer-readable storage medium), in which computer program instructions are stored. When the computer program instructions run on a computer, the computer is caused to execute the method for managing an adaptive parameter set as described in any one of the above embodiments.

[0350] Exemplarily, the above computer-readable storage medium may include, but is not limited to: magnetic storage devices (such as hard disks, floppy disks, or magnetic tapes, etc.), optical discs (such as Compact Disks (CDs), Digital Versatile Disks (DVDs), etc.), smart cards, and flash memory devices (such as Erasable Programmable Read-Only Memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0351] The embodiments of the present disclosure provide a computer program product containing instructions. When the computer program product runs on a computer, the computer is caused to execute the method for managing an adaptive parameter set as described in any one of the above embodiments.

[0352] As described above, the above are only the specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present disclosure should be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for managing an adaptive parameter set, characterized in that The method includes: Obtaining a target video category and / or target video features corresponding to a target video block; Determining a correction factor for each decision factor according to the target video category and / or the target video features, where one decision factor corresponds to one preset filtering mode, the decision factor is used to indicate whether to filter a video block, and when filtering, it indicates to use any one of a new APS, a historical APS in a historical candidate set, and a preset APS; Correcting each decision factor according to the target correction factor to obtain a plurality of corrected decision factors, where the probability of a corrected decision factor indicating filtering is greater than the probability of the decision factor before correction indicating filtering, and the probability of a corrected decision factor indicating using a new APS when filtering is greater than the probability of the decision factor before correction indicating using a new APS when filtering; Storing at least one obtained new APS into the historical candidate set according to each corrected decision factor.

2. The method according to claim 1, wherein The historical candidate set includes a plurality of historical APSs and a plurality of preset identifiers, where one preset identifier corresponds to at least one historical APS, and different preset identifiers are sorted in chronological order. The storing the at least one obtained new APS into the historical candidate set includes: Deleting all historical APSs corresponding to a first identifier in the historical candidate set, where the first identifier is an identifier in the plurality of preset identifiers, and the time corresponding to the first identifier is earlier than the time corresponding to any other identifier except the first identifier in the plurality of preset identifiers; Storing the at least one new APS into the historical candidate set, and updating the corresponding relationship between the preset identifier and the APS in the historical candidate set according to the time when the at least one new APS is stored into the historical candidate set, so that the time corresponding to a second identifier corresponding to the at least one new APS in the plurality of preset identifiers is later than the time corresponding to any other identifier except the second identifier in the plurality of preset identifiers.

3. The method according to claim 1 or 2, characterized in that, Obtaining the target video category corresponding to the target video block includes: Obtaining the target video content of the target video block; Determining at least one content feature of the target video content, where the content feature is an optical flow feature, a frequency domain feature, a spatial feature, or a depth feature; Determining the target video category according to the at least one content feature.

4. The method according to claim 1 or 2, characterized in that, The target video features include at least one of the following features: image resolution, texture complexity, motion complexity, color histogram.

5. The method according to claim 1 or 2, characterized in that The target video block is a video block in a video to be processed; The target video category is the video category of the video to be processed, and the target video features are the video features of the video to be processed, or The target video category is the video category of the target video block, and the target video features are the video features of the target video block.

6. The method according to claim 1 or 2, characterized in that The decision factor is the Lagrange factor in the rate-distortion optimization (RDO) decision process.

7. The method according to claim 1 or 2, characterized in that, The preset filtering mode is any one of the following adaptive loop filters (ALFs): chrominance ALF, luma ALF, cross-component adaptive loop filter (CCALF) for the difference between the blue component and luma (Cb), and CCALF for the difference between the red component and luma (Cr).

8. The method according to claim 1 or 2, characterized in that, The target video block is any one of the following types: sub-picture, tile, slice, coding tree unit (CTU), coding unit (CU).

9. An electronic device, characterized in that, Comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store instructions executable by the processor; when the processor executes the instructions, it executes the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and when the computer instructions run on a computer, the computer is caused to execute the method according to any one of claims 1-8.

Images