Loop filtering and coding method, electronic device, storage medium and program product

Abstract

Embodiments of the present application provide a loop filtering and coding method, an electronic device, a storage medium and a program product. The loop filtering method comprises: determining a corresponding reference image block of a to-be-processed image block in a reference frame adjacent to the to-be-processed image block in a time domain, and predicting a loop filtering opening probability of the to-be-processed image block according to a loop filtering result of the reference image block; determining spatial domain coding information of the to-be-processed image block, the spatial domain coding information being used to represent a texture complexity of the to-be-processed image block; and if the spatial domain coding information and the loop filtering opening probability meet a preset filtering condition, performing loop filtering decision on the to-be-processed image block. The scheme provided in the embodiments can improve the accuracy of the decision result by combining the time domain information and the spatial domain information to determine whether to perform loop filtering decision on the to-be-processed image, and in addition, can filter out part of the to-be-processed image blocks that do not need to be processed by loop filtering through the preset filtering condition, thereby reducing the computational complexity.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of image processing, and in particular to a loop filtering and coding method, an electronic device, a storage medium and a program product. BACKGROUND

[0002] In the field of video coding technology, in order to reduce compression distortion, adaptive loop filtering processing is generally performed on video frames to improve compression efficiency and reconstruction quality.

[0003] Generally, when adaptive loop filtering is performed, first, covariance information statistics is performed according to a decision result of whether to perform loop filtering on a plurality of image blocks, and a filter coefficient set of the plurality of image blocks is determined according to the covariance statistics result; then, the decision result of loop filtering of the image blocks is updated through rate-distortion optimization (RDO) decision according to the filter coefficient set, and then the step of performing covariance information statistics according to the decision result of loop filtering can be repeated for multiple iterations. The final decision result of loop filtering of the image blocks and the filter coefficient set can be obtained according to the multiple iteration results.

[0004] However, in the above process, the filter coefficient set needs to be calculated multiple times and the decision result of loop filtering of the image blocks needs to be updated, resulting in high computational complexity. SUMMARY

[0005] Therefore, embodiments of the present application provide a loop filtering scheme to at least partially solve the above problems.

[0006] According to a first aspect of embodiments of the present application, a loop filtering method is provided, comprising:

[0007] determining a reference image block corresponding to a to-be-processed image block in a reference frame adjacent to the to-be-processed image block in a time domain, and predicting a loop filtering on probability of the to-be-processed image block according to a loop filtering result of the reference image block;

[0008] determining spatial domain coding information of the to-be-processed image block, the spatial domain coding information being used to represent a texture complexity of the to-be-processed image block;

[0009] if the spatial domain coding information and the loop filtering on probability meet a preset filtering condition, performing loop filtering decision on the to-be-processed image block.

[0010] Optionally, in any embodiment of the present application, if the spatial domain coding information and the loop filtering on probability meet the preset filtering condition, performing loop filtering decision on the to-be-processed image block, comprises:

[0011] updating the loop filter on probability according to the spatial domain coding information;

[0012] If the updated loop filter on probability meets a preset probability condition, performing loop filter decision on the to-be-processed image block.

[0013] Or, if the updated loop filter on probability does not meet the preset probability condition, determining not to perform loop filter processing on the to-be-processed image block.

[0014] Optionally, in any of the embodiments of the present application, the updating of the loop filter on probability according to the spatial domain coding information comprises:

[0015] updating the loop filter on probability according to the temporal level of the video frame in which the to-be-processed image block is located and the spatial domain coding information.

[0016] Optionally, in any of the embodiments of the present application, the updating of the loop filter on probability according to the spatial domain coding information comprises:

[0017] inputting the spatial domain coding information and the loop filter on probability into a pre-trained machine learning model, and outputting the updated loop filter on probability through the machine learning model.

[0018] Optionally, in any of the embodiments of the present application, the spatial domain coding information comprises at least one of the following: a partition depth of the to-be-processed image block, a coding bit number of the to-be-processed image block, boundary strength information of the to-be-processed image block, image gradient information of the to-be-processed image block, boundary strength information of a reference block adjacent to the to-be-processed image block in spatial domain, and image gradient information of the reference block adjacent to the to-be-processed image block in spatial domain.

[0019] Optionally, in any of the embodiments of the present application, if the spatial domain coding information comprises at least two kinds, the updating of the loop filter on probability according to the spatial domain coding information comprises:

[0020] performing concatenated comparison according to preset parameter thresholds corresponding to the at least two kinds of spatial domain coding information.

[0021] updating the loop filter on probability according to the concatenated comparison result.

[0022] According to a second aspect of an embodiment of the present application, an encoding method is provided, comprising:

[0023] determining a reference image block corresponding to the to-be-processed image block in a reference frame adjacent to the to-be-processed image block in temporal domain, and predicting a loop filter on probability of the to-be-processed image block according to a loop filter result of the reference image block;

[0024] determining spatial domain coding information of the to-be-processed image block, the spatial domain coding information being used to represent texture complexity of the to-be-processed image block;

[0025] if the spatial domain coding information and the loop filter opening probability meet a preset filter condition, performing loop filter decision on the to-be-processed image block;

[0026] performing loop filter processing on the to-be-processed image block or not performing loop filter processing on the to-be-processed image block according to a loop filter decision result, to obtain an output image block;

[0027] encoding the output image block to obtain a bitstream carrying a loop filter processing identifier.

[0028] According to a third aspect of the embodiments of the present application, a decoding method is provided, including:

[0029] decoding a bitstream and performing video frame reconstruction to obtain a to-be-processed reconstructed block;

[0030] performing loop filter processing on the reconstructed block according to a loop filter processing identifier carried in the bitstream, wherein the loop filter processing identifier is determined by the decoding method.

[0031] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, including a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used to deposit at least one executable instruction, the executable instruction makes the processor execute corresponding operations of the above method.

[0032] According to a fifth aspect of the embodiments of the present application, a computer storage medium is provided, which stores a computer program, the program is executed by a processor to implement the above method.

[0033] According to the scheme provided by the embodiments of the present application, the spatial domain coding information can be used to represent the texture complexity of the to-be-processed image block, the loop filter opening probability obtained through the loop filter result of the reference image block in the time domain can be used to represent the time domain filter information of the to-be-processed image, and the combination of the time domain information and the spatial domain information can be used to determine whether to perform loop filter decision on the to-be-processed image, which can improve the accuracy of the decision result. In addition, by setting the preset filter condition corresponding to the spatial domain coding information and the loop filter opening probability, some to-be-processed image blocks that do not need to be processed by the loop filter can be filtered out through the preset filter condition, and the calculation complexity is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0035] Figure 1 A flowchart of a loop filtering decision-making process in general cases;

[0036] Figure 2 A flowchart of steps of a loop filtering method according to an embodiment of the present application;

[0037] Figure 3 A schematic diagram of a use scenario of a loop filtering method according to an embodiment of the present application;

[0038] Figure 4 A flowchart of steps of an encoding method according to an embodiment of the present application;

[0039] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those skilled in the art should belong to the scope of protection of the embodiments of the present application.

[0041] The specific implementation of the embodiments of the present application will be further described below in conjunction with the drawings of the embodiments of the present application.

[0042] Referring to Figure 1 , a flowchart of a loop filtering decision-making process in general cases is shown, as shown in the figure, taking an encoding process as an example, the loop filtering decision-making process can include the following steps:

[0043] 1) Covariance classification statistics

[0044] The covariance of a plurality of image groups (Coding Tree Unit, CTU) and the ALF (Adaptive Loop Filter) category of the smallest image block CU in the CTU are counted.

[0045] 2) Filter coefficient initialization

[0046] According to the covariance of the plurality of CTUs and the category of the minimum image block CU, the ALF filter coefficient set of the CTU is derived.

[0047] At this time, the iteration number is 0.

[0048] 3) CTU-level filter RDO decision

[0049] According to the ALF filter coefficient set, whether to enable ALF for each CTU is determined by RDO.

[0050] 4) Determine whether the iteration number reaches th0.

[0051] If not, continue to perform step 5, and if so, perform step 7.

[0052] 5) Increase the iteration number by one.

[0053] 6) Rederive the filter coefficient.

[0054] According to the "whether to enable ALF" updated in step 3, the covariance of the CTU enabling ALF filtering is re-counted, and the ALF filter coefficient set is re-derived according to the statistical information obtained by counting.

[0055] After step 6 is performed, return to step 3 to continue.

[0056] 7) Output each CTU ALF filter decision and the final ALF filter coefficient set. The ALF filter decision can be used to represent whether the CTU is subjected to loop filtering, and the ALF filter coefficient set can be used as a parameter for loop filtering.

[0057] However, in the above steps, the above steps 3-6 are looped for each CTU, resulting in high computational complexity. In addition, although not every CTU finally corresponds to an ALF filter decision of enabling ALF, in order to perform the RDO decision in step 3, the covariance of each CUT and the ALF category of the minimum image block CU in the CTU need to be counted, which increases the additional redundant calculation amount.

[0058] In order to reduce the computational complexity, another loop filtering scheme is provided in the embodiments of the present application, which will be described in detail below.

[0059] Referring to Figure 2 , Figure 2 is a step flow chart of a loop filtering method according to an embodiment of the present application. Specifically, the loop filtering method provided in the embodiment includes the following steps:

[0060] S201, determine a corresponding reference image block of the to-be-processed image block in a reference frame adjacent in the time domain, and predict the loop filter opening probability of the to-be-processed image block according to a loop filter result of the reference image block;

[0061] The to-be-processed image block can be any image block that needs to be processed by loop filtering in the image coding process, and the embodiment is not limited in this regard. The to-be-processed image block can be an image block divided into a bottom layer, or an image group composed of a plurality of bottom layers, and the embodiment is not limited in this regard.

[0062] Since the video includes a plurality of video frames with a time sequence, the video frame in which the to-be-processed image block is located has a reference frame adjacent in the time domain. The reference frame can be the first n frames and the last n frames on the time axis of the video frame, and n can be defined by a person skilled in the art according to the needs, and the embodiment is not limited in this regard.

[0063] The image block in the reference frame that has a texture similarity greater than a threshold with the to-be-processed image block can be used as the reference image block of the to-be-processed image. Alternatively, the image block in the reference frame that has the same position as the to-be-processed image can be used as the reference image block of the to-be-processed image. The specific standard for determining the reference image block can refer to the related technologies of the embodiment, and will not be described here.

[0064] After the reference image block is determined, the loop filter opening probability of the to-be-processed image block can be predicted from the loop filter result of the reference image block.

[0065] Specifically, the number of reference image blocks that have been processed by loop filtering and the number of reference image blocks that have not been processed by loop filtering can be counted to determine the loop filter opening probability of the to-be-processed image block. For example, the percentage of the number of reference image blocks that have been processed by loop filtering to the total number can be determined as the loop filter opening probability of the to-be-processed image block.

[0066] Further, the time axis distance between the reference image block and the to-be-processed image block can also be used to determine the loop filter opening probability of the to-be-processed image block. For example, the weight of the reference image block can be determined according to the time axis distance between the reference image block and the to-be-processed image block, and the loop filter processing result of the reference image block can be weighted and summed to obtain the loop filter opening probability of the to-be-processed image block.

[0067] The loop filter opening probability is used to represent the probability of processing the to-be-processed image block by loop filtering.

[0068] In addition, in the embodiment, when calculating the loop filter opening probability according to the loop filter result of the reference image block, the rate-distortion loss value RDCost before and after ALF filtering can also be used for calculation.

[0069] Specifically, the RDCost difference values of the accumulated video frames before and after the ALF filtering processing can be accumulated, and the RDCost difference values can be mapped to percentage influence factors according to a preset mapping relationship. After the loop filtering opening probability is calculated according to the loop filtering result of the reference image block, the percentage influence factors and the loop filtering opening probability can be used as the final calculated loop filtering opening probability. In this way, the result of the ALF filtering processing of the accumulated video frames can be further considered, and the accuracy of the calculated loop filtering opening probability is improved.

[0070] The filtering RDO decision of the frame with a smaller accumulated RDCost difference value is skipped according to the preset threshold. On the one hand, this scheme only considers the frame-level filtering decision, and does not more finely predict and skip the CTU-level filtering, resulting in a large loss of coding performance and the inability to save the redundant covariance calculation. On the other hand, this scheme only simply considers the time-domain statistical characteristics of the filtering decision, and ignores the spatial characteristics of the current frame, so that the filtering decision prediction accuracy is not high enough.

[0071] In S202, spatial domain encoding information of the to-be-processed image block is determined, and the spatial domain encoding information is used to represent the texture complexity of the to-be-processed image block.

[0072] In this embodiment, the spatial domain encoding information is the encoding information of the to-be-processed image block in the spatial domain. The spatial domain encoding information can be any information capable of representing the texture complexity of the to-be-processed image block, which is not limited in this embodiment.

[0073] For example, the spatial domain encoding information can include at least one of the following: a partition depth of the to-be-processed image block, a number of encoding bits of the to-be-processed image block, boundary strength information of the to-be-processed image block, image gradient information of the to-be-processed image block, boundary strength information of a reference block adjacent to the to-be-processed image block in the spatial domain, and image gradient information of the reference block adjacent to the to-be-processed image block in the spatial domain.

[0074] The partition depth of the to-be-processed image block is used to represent the minimum image block size when the to-be-processed image block is encoded. The larger the partition depth is, the smaller the minimum image block is, and the higher the texture complexity is.

[0075] A new generation of international video coding standard VVC (Versatile Video Coding) is based on image blocks. When using VVC as a standard for video coding, the image frame is generally divided into multiple image blocks, which can be, for example, Coding Tree Units (CTUs). The image blocks can then be divided by a quadtree to obtain first layer leaf nodes, and the first layer leaf nodes can be further divided to obtain multiple layer leaf nodes, each of which can correspond to a layer depth. After multiple divisions, the bottom layer leaf nodes, i.e., Coding Units (CUs), can be obtained. After the CUs are divided, residuals can be obtained by performing intra prediction or inter prediction on the CUs, and the CUs are encoded based on the residuals. The image block to be processed in the embodiment can be a CTU, and the level of the bottom layer leaf node corresponding to the CU after the CTU is divided is the division depth of the CTU.

[0076] The number of encoding bits of the image block to be processed refers to the number of bits of the image block to be processed after the image block to be processed is compressed and encoded. The higher the texture complexity, the more the number of encoding bits.

[0077] The boundary strength information of the image block to be processed is used to indicate the strength of the boundary in the image block to be processed. The higher the texture complexity, the higher the strength of the boundary.

[0078] The image gradient information of the image block to be processed is used to indicate the image gradient of the image block to be processed. The higher the texture complexity, the higher the image gradient.

[0079] The boundary strength information of the reference block adjacent to the image block to be processed in the spatial domain can be the boundary strength information of other image blocks adjacent to the position of the image block to be processed in the image frame. The higher the boundary strength of the other image blocks, the higher the texture complexity of the image block to be processed.

[0080] The image gradient information of the reference block adjacent to the image block to be processed in the spatial domain can be the image gradient information of other image blocks adjacent to the position of the image block to be processed in the image frame. The higher the image gradient of the other image blocks, the higher the texture complexity of the image block to be processed.

[0081] In S203, if the spatial domain encoding information and the loop filter opening probability meet the preset filter condition, loop filter decision is performed on the image block to be processed.

[0082] When performing loop filtering, the image block to be processed with a higher loop filter opening probability or a higher image texture complexity represented by the spatial domain encoding information is preferentially subjected to loop filtering. Therefore, the preset filter condition can be set according to this principle, so as to filter out some image blocks to be processed that do not need to be subjected to loop filtering.

[0083] The specific content of the preset filtering condition can be determined by those skilled in the art according to requirements, and the embodiment is not limited in this regard.

[0084] Optionally, in the embodiment, the step S203 can include: updating the loop filter opening probability according to the spatial domain coding information; if the updated loop filter opening probability meets a preset probability condition, making a loop filter decision on the to-be-processed image block; and the method further includes: if the updated loop filter opening probability does not meet the preset probability condition, determining not to perform loop filter processing on the to-be-processed image block.

[0085] In the embodiment, by updating the loop filter opening probability according to the spatial domain coding information, the updated loop filter opening probability can represent both the time domain information and the spatial domain information, so that the preset filtering condition can be a preset probability condition, greatly reducing the complexity of the preset filtering condition and improving the processing efficiency.

[0086] In one implementation manner of the application, the spatial domain coding information and the loop filter opening probability can be input into a pre-trained machine learning model, and the updated loop filter opening probability is output by the machine learning model. By updating the loop filter opening probability through the pre-trained machine learning model, complex judgment logic is not required, and the accuracy of the updated loop filter opening probability can be improved, that is, the updated loop filter opening probability can more accurately represent the probability of performing loop filtering on the to-be-processed image block.

[0087] In another implementation manner of the application, if the spatial domain coding information includes at least two types, a cascade comparison can be performed according to preset parameter thresholds corresponding to the at least two types of spatial domain coding information, and the loop filter opening probability is updated according to the cascade comparison result. By setting multiple comparison conditions, the updating of the loop filter opening probability can be quickly implemented, and the processing speed is improved.

[0088] For example, the first level can correspond to the first spatial domain coding information, and there is a first threshold value corresponding thereto. By comparing the first spatial domain coding information and the first threshold value, if the first spatial domain coding information is smaller than the first threshold value, the loop filter opening probability can be set to zero; if the first spatial domain coding information is not smaller than the first threshold value, the next level of judgment can be continued.

[0089] The second level can correspond to the second spatial domain coding information, and there is a second threshold value corresponding thereto. By comparing the second spatial domain coding information and the second threshold value, if the second spatial domain coding information is smaller than the second threshold value, the next level of judgment can be continued, and the loop filter opening probability is set to zero or remains unchanged through the next level of judgment; if the second spatial domain coding information is not smaller than the second threshold value, the next level of judgment can be continued, and the loop filter remains unchanged or is set to one through the next level of judgment.

[0090] Optionally, in the embodiment, the loop filter opening probability can also be updated according to the time domain level of the video frame in which the image block to be processed is located and the spatial domain coding information.

[0091] The time domain level of the video frame in which the image block to be processed is located is used to represent the level in time of the image frame. Specifically, in the H.264 SVC standard, the time scalability is realized by setting discardable reference frames in the GOP (Group of Pictures). The 0th layer can be independently coded; the 1st layer is coded in dependence on the 0th layer; the 2nd layer is coded in dependence on the 1st layer; and the decoding result of the high layer video frame does not affect the decoding of the low layer video frame. The more the number of layers to be decoded, the higher the video frame rate obtained by decoding. Since the decoding result of the high layer video frame does not affect the decoding of the low layer video frame, the quality of the high layer decoding can be lower, and the probability of performing the loop filter processing can also be lower. By introducing the time domain level, the loop filter decision on the image block to be processed in the high layer time domain level can be skipped as much as possible, and the processing speed is improved.

[0092] When the loop filter opening probability is updated by the cascade comparison, the time domain level can also correspond to a level. For example, a level threshold corresponding to the time domain level can be set, and it is determined whether the time domain level of the image block to be processed is less than the level threshold. If yes, the next level comparison can be continued, and the loop filter opening probability is set to zero or remains unchanged according to the next level comparison result. If no, the next level comparison can be continued, and the loop filter remains unchanged or is set to one according to the next level comparison result.

[0093] The scheme provided in the embodiment can be used to represent the texture complexity of the image block to be processed by the spatial domain coding information, and the time domain filtering information of the image to be processed can be represented by the loop filter opening probability obtained by the loop filter result of the reference image block in the time domain. By combining the time domain information and the spatial domain information to determine whether the image to be processed is subjected to the loop filter decision, the accuracy of the decision result can be improved. In addition, by setting the preset filtering condition corresponding to the spatial domain coding information and the loop filter opening probability, some image blocks to be processed which do not need to be subjected to the loop filter processing can be filtered out by the preset filtering condition, and the calculation complexity is reduced.

[0094] Reference Figure 3 , Figure 3 FIG. 1 is a use scenario diagram of a loop filter method according to an embodiment of the present application. Specifically, the loop filter method provided in the embodiment includes the following steps:

[0095] S301, a reference image block corresponding to the image block to be processed in the adjacent reference frame in the time domain is determined, and the initial loop filter opening probability prob of the image block to be processed is predicted according to the loop filter result of the reference image block.

[0096] In the embodiment, the ALF opening conditions of 9 adjacent reference image blocks CTUs on the time domain adjacent frames can be counted to obtain the loop filter opening probability prob of the image block to be processed.

[0097] In S302, it is judged whether the time domain level of the image block to be processed is greater than or equal to a preset level threshold th1.

[0098] In the embodiment, the level threshold th1 can be set as the top time domain level number.

[0099] If not, the step S303 is continued to be executed, and if yes, the step S304 is continued to be executed.

[0100] In S303, it is judged whether the partition depth of the image block to be processed is less than or equal to a preset second threshold th2.

[0101] If yes, the step S305 is continued to be executed, and if yes, the step S306 is continued to be executed.

[0102] In the embodiment, the second threshold th2 can be set as 1, that is, the image to be processed is partitioned once.

[0103] In S304, it is judged whether the encoding bit number of the image block to be processed is less than a third threshold th3.

[0104] If less than, the loop filter opening probability is updated as 0; and if not less than, the loop filter opening probability is maintained unchanged, and the step S307 is continued to be executed.

[0105] In S305, it is judged whether the encoding bit number of the image block to be processed is less than a fourth threshold th4.

[0106] If less than, the loop filter opening probability is updated as 0; and if not less than, the loop filter opening probability is maintained unchanged, and the step S307 is continued to be executed.

[0107] In S306, it is judged whether the encoding bit number of the image block to be processed is less than a fifth threshold th5.

[0108] If less than, the loop filter opening probability is maintained unchanged; and if not less than, the loop filter opening probability is updated as 1, and the step S307 is continued to be executed.

[0109] It should be noted that the third threshold th3, the fourth threshold th4 and the fifth threshold th5 can be the same or different, and are within the protection scope of the present application. In the embodiment, the third threshold th3, the fourth threshold th4 and the fifth threshold th5 are set as 100, that is, the threshold corresponding to the encoding bit number of the image block to be processed is 100 bits.

[0110] In this embodiment, the time domain level is taken as the first level, the division depth is taken as the second level, and the number of coding bits is taken as the third level, so that the loop filter opening probability can be updated by cascading comparison.

[0111] S307, for the image block to be processed, if the loop filter opening probability p is greater than the sixth threshold th6, performing loop filter decision.

[0112] There will be a condition in the dashed box in the figure, and the loop filter opening probability of the image block to be processed is determined, and step S307 can be continued according to the determined loop filter opening probability p.

[0113] If the loop filter opening probability is not greater than the sixth threshold, it is determined that the loop filter processing is not performed, and the process can be ended.

[0114] In this embodiment, in order to balance the acceleration ratio and performance loss, th6 is 0.6.

[0115] The method of performing loop filter decision can refer to related technologies, for example, the related description in Figure 1 , which will not be described here.

[0116] Referring to Figure 4 , Figure 4 is a step flowchart of an encoding method according to an embodiment of the present application. Specifically, the method provided in this embodiment includes the following steps:

[0117] S401, determining a reference image block corresponding to the image block to be processed in a reference frame adjacent to the image block to be processed in the time domain, and predicting the loop filter opening probability of the image block to be processed according to the loop filter result of the reference image block;

[0118] S402, determining the spatial domain coding information of the image block to be processed, the spatial domain coding information being used to represent the texture complexity of the image block to be processed;

[0119] S403, if the spatial domain coding information and the loop filter opening probability meet a preset filter condition, performing loop filter decision on the image block to be processed;

[0120] S404, according to the loop filter decision result, performing loop filter processing on the image block to be processed or not performing loop filter processing on the image block to be processed, to obtain an output image block;

[0121] S405, encoding according to the output image block to obtain a code stream carrying a loop filter processing identifier.

[0122] The specific implementation of each step in this embodiment can refer to the corresponding description in the corresponding steps and units of the method embodiments described above, and has corresponding beneficial effects, which will not be repeated here. The skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the devices and modules described above can refer to the corresponding process description in the foregoing method embodiments, which will not be repeated here.

[0123] In this embodiment, the spatial domain coding information can be used to represent the texture complexity of the to-be-processed image block, and the loop filter opening probability obtained by the loop filter result of the reference image block in the time domain can be used to represent the time domain filtering information of the to-be-processed image. By combining the time domain information and the spatial domain information to determine whether to perform loop filtering on the to-be-processed image, the accuracy of the decision result can be improved. In addition, by setting the preset filtering condition corresponding to the spatial domain coding information and the loop filter opening probability, some to-be-processed image blocks that do not need to be processed by the loop filter can be filtered out through the preset filtering condition, thereby reducing the computational complexity.

[0124] Another embodiment of the present application provides a step flowchart of a decoding method. Specifically, the method provided in this embodiment includes the following steps:

[0125] Decoding the code stream and performing video frame reconstruction to obtain a to-be-processed reconstruction block.

[0126] According to the loop filter processing identifier carried in the code stream, performing loop filter processing on the reconstruction block, wherein the loop filter processing identifier is determined by the encoding method provided in the above embodiment.

[0127] The specific implementation of each step in this embodiment can refer to the corresponding description in the corresponding steps and units of the method embodiments described above, and has corresponding beneficial effects, which will not be repeated here. The skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the devices and modules described above can refer to the corresponding process description in the foregoing method embodiments, which will not be repeated here.

[0128] The loop filtering scheme or the coding scheme provided by any of the above embodiments of the present application can be executed by a video encoding end (encoder) when encoding a video stream, to improve the processing efficiency of the encoding process. It can be applied to various different scenarios, such as storage and streaming of regular video games, specifically: each image block in a video frame can be subjected to loop filtering processing by the loop filtering scheme provided by the embodiments of the present application, and a loop filtering processing identifier is added to the generated video code stream, for storage and transmission in a video stream service or other similar application; for example, a low-latency scenario such as a video conference, a video live broadcast, specifically: conference video data can be collected by a video collection device, and each image block in a video frame can be subjected to loop filtering processing by the method provided by the embodiments of the present application, and a loop filtering processing identifier is added to the generated video code stream, and is sent to a conference terminal, and the corresponding conference video picture is obtained by decoding the video code stream according to the loop filtering identifier by the conference terminal; for example, a virtual reality scenario, each image block in a video frame can be subjected to loop filtering processing by the method provided by the embodiments of the present application, and a loop filtering processing identifier is added to the generated video code stream, and is sent to a virtual reality related device (such as a VR virtual glasses, etc.), and the corresponding video picture is obtained by decoding the video code stream according to the loop filtering identifier by the VR device, and the corresponding VR function is realized based on the video picture, and the like.

[0129] Referring to Figure 5 , a structural schematic diagram of an electronic device according to Embodiment Four of the present application is shown, and the specific implementation of the electronic device is not limited by the specific embodiments of the present application.

[0130] As shown in Figure 5 , the electronic device can include a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0131] Among them:

[0132] The processor 502, the communications interface 504, and the memory 506 complete mutual communication through the communications bus 508.

[0133] The communications interface 504 is configured to communicate with other electronic devices or servers.

[0134] The processor 502 is configured to execute the program 510, and specifically can execute the related steps in the above loop filtering method or coding method embodiments.

[0135] Specifically, the program 510 can include program code, and the program code includes computer operation instructions.

[0136] The processor 502 can be a CPU, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors of the smart device can be of the same type, such as one or more CPUs; or can be of different types, such as one or more CPUs and one or more ASICs.

[0137] The memory 506 is configured to store a program 510. The memory 506 can include a high-speed RAM memory, and can further include a non-volatile memory, such as at least one disk memory.

[0138] The program 510 can include a plurality of computer instructions, and the program 510 can specifically cause the processor 502 to perform operations corresponding to the loop filtering method or the coding method described in any of the foregoing method embodiments by means of the plurality of computer instructions.

[0139] The specific implementation of each step in the program 510 can refer to the corresponding description of the corresponding steps and units in the foregoing method embodiments, and has corresponding beneficial effects, which will not be described here. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working process of the device and the module described above can refer to the corresponding process description in the foregoing method embodiments, which will not be described here.

[0140] The embodiments of the present application further provide a computer storage medium, which stores a computer program. The program is executed by a processor to implement the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk, or a magneto-optical disk, etc.

[0141] The embodiments of the present application further provide a computer program product, which includes computer instructions. The computer instructions instruct a computing device to perform operations corresponding to any of the foregoing method embodiments.

[0142] In addition, it should be noted that the information related to the user (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to sample data for training the model, data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the country and region, and provide corresponding operation portal for the user to choose authorization or refusal.

[0143] It should be noted that, according to the needs of implementation, each component / step described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or part of the operation of the components / steps can be combined into a new component / step, to achieve the purpose of the embodiments of the present application.

[0144] The above method according to the embodiments of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk, or be implemented by computer code originally stored in a remote recording medium or non-transitory machine readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing of recording medium using a general computer, special processor or programmable or special hardware such as Application Specific Integrated Circuit (ASIC) or Field Programmable Gate Array (FPGA). It can be understood that the computer, processor, microprocessor controller or programmable hardware includes storage components (for example, Random Access Memory (RAM), Read-Only Memory (ROM), flash memory, etc.) that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method described herein is implemented. In addition, when the general computer accesses the code for implementing the method shown herein, the execution of the code will convert the general computer into a special computer for executing the method shown herein.

[0145] Those skilled in the art can understand that the units and method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered as beyond the scope of the embodiments of the present application.

[0146] The above embodiments are only used to illustrate but not to limit the embodiments of the present application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, and all equivalent technical solutions belong to the scope of the embodiments of the present application. The patent protection scope of the embodiments of the present application should be defined by the claims.

Claims

1. A loop filtering method, comprising: determining a corresponding reference image block of a to-be-processed image block in a reference frame adjacent to the to-be-processed image block in a time domain, and predicting a loop filtering on probability of the to-be-processed image block according to a loop filtering result of the corresponding reference image block; determining spatial domain coding information of the to-be-processed image block, the spatial domain coding information being used to represent texture complexity of the to-be-processed image block; if the spatial domain coding information and the loop filtering on probability meet a preset filtering condition, making a loop filtering decision on the to-be-processed image block.

2. The method of claim 1, wherein, The if the spatial domain coding information and the loop filtering on probability meet a preset filtering condition, making a loop filtering decision on the to-be-processed image block, comprises: updating the loop filtering on probability according to the spatial domain coding information; if the updated loop filtering on probability meets a preset probability condition, making a loop filtering decision on the to-be-processed image block; The method further comprises: if the updated loop filtering on probability does not meet the preset probability condition, determining not to make a loop filtering processing on the to-be-processed image block.

3. The method of claim 2, wherein, The updating the loop filtering on probability according to the spatial domain coding information, comprises: updating the loop filtering on probability according to a time domain level of a video frame in which the to-be-processed image block is located and the spatial domain coding information.

4. The method of claim 2, wherein, The updating the loop filtering on probability according to the spatial domain coding information, comprises: inputting the spatial domain coding information and the loop filtering on probability into a pre-trained machine learning model, and outputting the updated loop filtering on probability through the machine learning model.

5. The method of claim 2, wherein, The spatial domain coding information comprises at least one of the following: a partition depth of the to-be-processed image block, a coding bit number of the to-be-processed image block, boundary strength information of the to-be-processed image block, image gradient information of the to-be-processed image block, boundary strength information of a reference block adjacent to the to-be-processed image block in a spatial domain, and image gradient information of the reference block adjacent to the to-be-processed image block in the spatial domain.

6. The method of claim 5, wherein, If the spatial domain coding information comprises at least two kinds, the updating the loop filtering on probability according to the spatial domain coding information, comprises: performing cascaded comparison according to preset parameter thresholds corresponding to the at least two kinds of spatial domain coding information; updating the loop filtering on probability according to a cascaded comparison result.

7. An encoding method, comprising: determining a corresponding reference image block of a to-be-processed image block in a reference frame adjacent to the to-be-processed image block in a time domain, and predicting a loop filtering on probability of the to-be-processed image block according to a loop filtering result of the corresponding reference image block; determining spatial domain coding information of the to-be-processed image block, the spatial domain coding information being used to represent texture complexity of the to-be-processed image block; if the spatial domain coding information and the loop filtering on probability meet a preset filtering condition, making a loop filtering decision on the to-be-processed image block; performing loop filtering processing on the to-be-processed image block or not performing loop filtering processing on the to-be-processed image block according to a loop filtering decision result, to obtain an output image block; performing encoding according to the output image block, to obtain a bitstream carrying a loop filtering processing identifier.

8. A decoding method, comprising: decoding a code stream and performing video frame reconstruction to obtain a to-be-processed reconstructed block; performing loop filtering processing on the reconstructed block according to a loop filtering processing flag carried in the code stream, wherein the code stream is obtained by the method in claim 7.

9. An electronic device comprising: a processor, a memory, a communication interface, and a communication bus, the processor, the memory, and the communication interface performing communication with each other through the communication bus; the memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the method in any one of claims 1-8.

10. A computer storage medium having a computer program stored thereon, the program being executed by a processor to implement the method in any one of claims 1-8.

Images