Filtering method and apparatus for av1

Abstract

The application discloses a filtering method of AV1, comprising the following steps: determining a to-be-filtered partition, wherein the to-be-filtered partition comprises a plurality of blocks, and the plurality of blocks form a plurality of rows; performing parallel filtering operations on at least two blocks in the plurality of blocks through a plurality of threads; wherein the plurality of threads are responsible for filtering operations of a plurality of rows in a one-to-one correspondence, the plurality of rows comprise at least two continuous rows in the plurality of rows, and each block in the at least two blocks is located in a different row. The application also discloses a filtering device, a computer device and a computer readable storage medium. The technical scheme provided by the application has the following advantages: compared with a filtering mode of performing filtering on blocks one by one through a single thread, the application adopts a plurality of threads, and under the premise of meeting filtering rules, two or more threads can be used to perform parallel filtering operations on different blocks, so that the efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video coding, and in particular to a filtering method and device of AV1, a computer device and a computer readable storage medium. BACKGROUND

[0002] AOMedia Video 1 (AV1) is an open video coding format designed for over-the- top streaming. AV1 is a successor to VP9 developed by the Alliance for Open Media (AOMedia). Many components of the AV1 project originated in earlier research efforts by Alliance members. Individual contributors started experimenting with technology platforms early: Xiph / Mozilla's Daala released code in 2010, Google's experimental VP9 evolution project VP10 on September 12, 2014, and Cisco's Thor on August 11, 2015. AV1 is based on the codebase of VP9, combined with other technologies, some of which were developed in these experimental formats. The first version 0.1.0 of the AV1 reference codec was released on April 7, 2016. The Alliance announced the release of the AV1 bitstream specification, and software-based reference encoder and decoder on March 28, 2018. On June 25, 2018, version 1.0.0 of the specification was released, verified. On January 8, 2019, the verified version 1.0.0 with specification errata 1 was released. The AV1 bitstream specification includes a reference video codec. With optimizations for AV1 encoders, AV1 can achieve higher compression efficiency than VP9 and H.264.

[0003] The encoding process of AV1 generally includes operations such as block division, prediction (inter prediction, intra prediction), data transformation, quantization, entropy coding and filtering. However, the current filtering method restricts the improvement of coding efficiency to some extent. SUMMARY

[0004] An embodiment of the present application provides a filtering method, device, computer device and computer readable storage medium of AV1, which can be used to solve the problems described above.

[0005] An embodiment of the present application provides a filtering method of AV1, comprising:

[0006] determining a to-be-filtered partition, the to-be-filtered partition comprising a plurality of blocks, the plurality of blocks forming a plurality of rows;

[0007] performing parallel filtering operations on at least two of the plurality of blocks by a plurality of threads; wherein the plurality of threads are responsible for filtering operations of a plurality of rows one by one, the plurality of rows comprising at least two consecutive rows in the plurality of rows, and each of the at least two blocks is located in a different row.

[0008] Optionally, the plurality of rows comprises a first row and one or more subsequent rows; and the performing parallel filtering operations on at least two of the plurality of blocks comprises:

[0009] for the first row, performing filtering operations in a sequence from left to right on each block of the first row;

[0010] for the one or more subsequent rows, performing filtering operations in a sequence from left to right on each block of a target row, wherein a filtering progress of the target row lags behind a filtering progress of a previous row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows.

[0011] Optionally, the plurality of rows comprises an mth row and one or more subsequent rows, m being an integer greater than or equal to 2; and the performing parallel filtering operations on at least two of the plurality of blocks comprises:

[0012] performing filtering operations in a sequence from left to right on each block of a target row by a thread assigned to the target row;

[0013] wherein a filtering progress of the target row lags behind a filtering progress of a previous row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows.

[0014] Optionally, the target row corresponds to an nth row, n being an integer greater than or equal to 2; and the performing parallel filtering operations on at least two of the plurality of blocks comprises:

[0015] recording a number of filtered blocks of each row, the filtering data identifying a number of blocks that have been filtered currently;

[0016] in a case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, performing filtering on a block in the nth row that has not been filtered currently, wherein the maximum preset value is a total number of blocks in the (n-1)th row;

[0017] in a case where the number of filtered blocks is greater than 2 and less than or equal to the maximum preset value, performing filtering on a corresponding block in the nth row, the corresponding block being located at a position of the number of filtered blocks minus 2 in the nth row.

[0018] Optionally, the method further comprises:

[0019] in a case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, configuring threads of the (n-1)th row to perform filtering operations on the (n+N-1)th row, N being a positive integer and used to represent the total number of the threads.

[0020] An aspect of an embodiment of the present application further provides a filtering device of AV1, comprising:

[0021] a determining module configured to determine a to-be-filtered partition, the to-be-filtered partition comprising a plurality of blocks, the plurality of blocks forming a plurality of rows;

[0022] a filtering module configured to perform parallel filtering operations on at least two blocks in the plurality of blocks through a plurality of threads, wherein the plurality of threads are responsible for filtering operations of a plurality of rows in a one-to-one correspondence, the plurality of rows comprising at least two consecutive rows in the plurality of rows, and each of the at least two blocks being located in a different row.

[0023] Optionally, the filtering module is further configured to:

[0024] record the number of filtered blocks of each row, the filtering data identifying the number of blocks that have been filtered at present;

[0025] in a case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, performing filtering on blocks in the nth row that have not been filtered at present, wherein the maximum preset value is the total number of blocks in the (n-1)th row, and n is an integer greater than or equal to 2;

[0026] in a case where the number of filtered blocks is greater than 2 and less than or equal to the maximum preset value, performing filtering on a corresponding block in the nth row, the position of the corresponding block in the nth row being the number of filtered blocks minus 2.

[0027] Optionally, the device further comprises a configuring module configured to:

[0028] in a case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, configuring threads of the (n-1)th row to perform filtering operations on the (n+N-1)th row, N being a positive integer and used to represent the total number of the threads.

[0029] An aspect of an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor being configured to implement the steps of the filtering method of AV1 when executing the computer program.

[0030] An aspect of the embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable by at least one processor to enable the at least one processor to perform the steps of the AV1 filtering method.

[0031] The AV1 filtering method, the device, the computer device and the computer readable storage medium provided by the embodiment of the present application have the following advantages.

[0032] Compared with the filtering method of filtering one block at a time by using a single thread, the embodiment uses a plurality of threads, and under the premise of meeting the filtering rule, two or more threads can be used to perform parallel filtering operation on different blocks, so that the efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The conventional filtering method of AV1 is schematically shown.

[0034] Figure 2 An application environment diagram of the AV1 filtering method according to the embodiment of the present application is schematically shown.

[0035] Figure 3 A flowchart of the AV1 filtering method according to the first embodiment of the present application is schematically shown.

[0036] Figure 4 A partition including a plurality of blocks is schematically shown.

[0037] Figure 5 A filtering order of the plurality of blocks is schematically shown.

[0038] Figure 6 Coordinates of the plurality of blocks are schematically shown.

[0039] Figure 7 An operation flow of the AV1 filtering method according to the first embodiment of the present application in an exemplary application is schematically shown.

[0040] Figure 8 A block diagram of the AV1 filtering device according to the second embodiment of the present application is schematically shown.

[0041] Figure 9 A hardware architecture diagram of the computer device suitable for implementing the AV1 filtering method according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0043] It should be noted that the descriptions involving "first", "second" and the like in the embodiments of the present application are only for the purpose of description and should not be understood as indicating or implying the relative importance of the technical features indicated or implying the number of technical features indicated. Therefore, the features limited by "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0044] In the description of the present application, it should be understood that the numerical reference number before the step does not indicate the order of execution of the steps before and after the step, but is only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as limiting the present application.

[0045] The following is the explanation of the terms of the present application:

[0046] AV1 (AOMedia Video 1): is an open source and royalty-free video codec developed by "Alliance for Open Media (AOMedia)". According to the usage, AV1 can achieve higher compression efficiency than VP9 and H.264.

[0047] In order to facilitate those skilled in the art to understand the technical solutions provided by the embodiments of the present application, the related art will be described as follows:

[0048] The AV1 encoding process includes the following processes: block partitioning, prediction, transformation, quantization, entropy encoding, filtering and post-processing, etc.

[0049] (1) Block partitioning:

[0050] The image (frame) can be partitioned (partitioning) into coding units (such as super blocks of 128x128 pixels) that are interlaced, adjacent and the same size, and then the image is processed in units of coding units. Among them, the super block can be divided into smaller blocks according to different partition modes, for example: the super block can be divided into a plurality of 4x4 pixel blocks. The partition mode can be four equal parts (SPLIT) or two equal parts (HORZ, VERT).

[0051] (2) Prediction:

[0052] According to the correlation between image pixels, the current pixel is predicted by using adjacent pixels. For example, by using intra prediction and inter prediction, the difference of each image is obtained according to the key frame, so as to reduce the amount of stored encoding information. Intra prediction is used to remove the spatial redundancy in the frame to obtain a residual unit with smaller pixel value than the coding unit. Inter prediction is used to remove the temporal redundancy between frames to obtain a residual unit with smaller pixel value than the coding unit.

[0053] Among them, the intra prediction (Intra Prediction) can predict the pixels of the target block according to the available information in the current frame. In most cases, the intra prediction is constructed from the adjacent pixels above and on the left of the target block to be predicted.

[0054] (3) Data transformation:

[0055] The low-frequency information and high-frequency information are separated by DCT (Discrete Cosine Transform) and the like, and the residual unit is transformed into a transform unit (TU). It should be noted that other transformation methods can also be used.

[0056] (4) Quantization:

[0057] Based on the quantization step, the transform coefficients in the TU are quantized to obtain quantization levels, and the unimportant data is zeroed to reduce the data amount.

[0058] (5) Entropy coding:

[0059] In the encoding process, the encoding according to the entropy principle does not lose any information, such as using the number of repetitions to represent the continuous repeated data.

[0060] (6) Filtering and post-processing: By filtering operation, the block effect and noise can be eliminated, and various filters can be used to achieve it.

[0061] The inventor understands that the current AV1 filtering method is to filter each block in turn from left to right and from top to bottom according to the coordinates of the block. As shown in Figure 1 Each block is processed row by row (blocks 0-5 are processed one by one first, then blocks 6-11 are processed one by one, and so on). If block 12 needs to be processed, it needs to wait for blocks 0-11 to be processed. It can be seen that the later blocks need to wait for a longer time, and the encoding efficiency is low.

[0062] Therefore, the present application aims to provide a filtering scheme of AV1. The scheme proposes a parallel filtering manner, which can solve the problem of long waiting time and low coding efficiency caused by the above-mentioned row-by-row filtering. For example, multiple threads can be started, assuming that the number of threads is N (a positive integer greater than 2), and each thread is responsible for the filtering operation of a row; filtering can also be performed in a manner that each row lags behind the previous row by 2 blocks. When the nth row starts processing the filtering operation of the b+2th block, the n+1th row starts the filtering of the bth block. For details, see below.

[0063] The following provides an exemplary application environment of the present application, for example, which can be used in a computer device 10000 as shown in Figure 1 The computer device 10000 can be configured to access the content (such as video) and services of a server.

[0064] The computer device 10000 can be configured to access the content (such as video) and services of a server.

[0065] The computer device 10000 can include an electronic device carrying or externally connected to a display panel, such as a mobile device, a tablet device, a laptop computer, a workstation, a virtual reality device, a game device, a digital streaming media device, a vehicle user terminal, a smart television, a set-top box, etc., and can also include a virtualized computing instance. The virtualized computing instance can include a virtual machine, such as a computer system, an operating system, a server, etc.

[0066] The computer device 10000 can be associated with one or more users. A single user can also use one or more of the computer devices 10000 to access a server. The computer device 10000 can travel to various locations and use different networks to access a server. The computer device 10000 can include multiple client programs, such as a video codec for providing encoding and decoding services. The video codec can encode and compress a video or image to facilitate transmission or storage of the video or image.

[0067] The following will provide a plurality of embodiments under the above-mentioned exemplary application environment to illustrate the filtering scheme of AV1.

[0068] Embodiment One

[0069] It should be noted that the execution subject of the present embodiment can be the computer device 10000.

[0070] The filtering operation described in the present embodiment can improve the output quality and improve the visual experience. The filtering operation can be implemented through a filter. The filter can be a specification or non-specification. Among them, the specification filter is a necessary part of the codec, and if it is missing, the video cannot be correctly decoded. The non-specification filter is an optional item.

[0071] Filters can be classified according to their application location. For example, pre-processing filters applied to the input before the encoding starts, post-processing filters applied to the output after the decoding is completed, and in-loop filters that are integrated as part of the encoding process in the encoding loop. Pre-processing and post-processing filters are usually non-normative and outside the codec. By definition, in-loop filters should be normative and part of the codec itself; they are used in the encoding optimization process and applied to the stored reference frames or inter-coded frames.

[0072] The filtering operations described in the embodiments can be applied in modules of in-loop filtering and post-processing.

[0073] The filtering operations can eliminate blockiness and noise, and exemplary filters that can be used include, but are not limited to:

[0074] Deblocking filter, performed at the 128x128 superblock level, and filtered separately for vertical and horizontal edges. For a 128x128 pixel superblock, first filter the vertical / horizontal edges aligned with each 8x8 block. If a 4x4 pixel transform is used, further filter the internal edges aligned with 4x4 pixel blocks.

[0075] Constrained directional enhancement filter, which eliminates or reduces the underlying noise and ringing effects near a hard edge of an image, while not blurring or damaging the edge. The edge direction search is performed at the 8x8 block level. There are a total of eight edge directions.

[0076] In-loop restoration filter, which can include separable symmetric Wiener filter, double self-guided filter, etc. The in-loop restoration filter can remove blurring ringing caused by block processing. Ringing effect is one of the factors affecting the quality of the restored image, which is caused by the selection of an inappropriate image model in image restoration. The causes of ringing effect include the loss of information (such as high-frequency information) in the image degradation process, which severely degrades the quality of the restored image and makes it difficult to perform subsequent processing on the restored image.

[0077] Figure 3 A flowchart of a filtering method of AV1 according to Embodiment One of the present application is schematically shown.

[0078] As shown in Figure 3 , the filtering method of AV1 can include steps S300-S302, wherein:

[0079] Step S300, determine a to-be-filtered partition, the to-be-filtered partition including a plurality of blocks, the plurality of blocks forming a plurality of rows.

[0080] The frame to be encoded (the partition to be filtered) can be divided into a plurality of superblocks (128x128 pixels). Each superblock is further divided into smaller blocks. In the filtering operation, each block can be filtered according to certain rules in units of these blocks.

[0081] In the present embodiment, the plurality of blocks into which the partition to be filtered is divided are arranged in rows, i.e. form a plurality of rows. As shown in the figure, the partition to be filtered includes 24 blocks: block 0 - block 23. Among them, block 0 - block 5 form the first row, block 6 - block 11 form the second row, block 12 - block 17 form the third row, and block 18 - block 23 form the fourth row. Figure 3

[0082] In step S302, parallel filtering operations are performed on at least two blocks in the plurality of blocks by a plurality of threads. Each of the plurality of threads is responsible for the filtering operation of a plurality of rows, the plurality of rows including at least two consecutive rows in the plurality of rows, and each of the at least two blocks is located in a different row.

[0083] Based on the filtering direction and rules of the filter, the blocks on the top edge can be filtered directly from left to right block by block. The blocks on the left edge can be filtered when the block directly above it is filtered. The blocks outside the top edge and the left edge can be filtered when the block directly above it and the block directly to the left of it are both filtered.

[0084] The following will be illustrated by way of example. Figure 4 In theory, the following applies:

[0085] When the filtering of block 0 is completed by thread #1, the filtering of block 1 is performed next. At this time, block 0 directly above block 6 is filtered, so block 6 can also be filtered at this time. Therefore, at this time, the filtering of block 1 can be performed by thread #1 and the filtering of block 6 can be performed by thread #2, thereby realizing parallel filtering of block 1 and block 6.

[0086] When the filtering of block 0 and block 1 is completed by thread #1 and the filtering of block 6 is completed by thread #2, the filtering of block 2 is performed by thread #1 next. At this time, block 1 directly above block 7 is filtered and block 6 directly to the left of block 7 is filtered, so block 7 can also be filtered at this time. Therefore, at this time, the filtering of block 2 can be performed by thread #1 and the filtering of block 7 can be performed by thread #2, thereby realizing parallel filtering of block 2 and block 7.

[0087] Therefore, to filter the blocks 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, the following applies: Figure 4 ​In the 24 block filtering is completed, the traditional filtering mode needs to be carried out 24 times of filtering operation according to the time sequence, corresponding to 24 filtering time units. And the multi-thread mode of the embodiment is adopted, some blocks can be filtered at the same time, so the time required for filtering the 24 blocks is less than 24 filtering time units. Therefore, the technical scheme of the embodiment can improve the filtering efficiency and reduce the time required for filtering.

[0088] Compared with the single-thread block-by-block filtering mode, the embodiment adopts several threads, and under the premise of meeting the filtering rules, two or more threads can be used for parallel filtering operation on different blocks, thereby improving the efficiency.

[0089] In an optional embodiment, the plurality of rows include a first row and one or more subsequent rows. The step S302 of performing parallel filtering operation on at least two blocks in the plurality of blocks can include:

[0090] For the first row, the following filtering operation is performed: filtering each block of the first row from left to right in turn;

[0091] For the one or more subsequent rows, the following filtering operation is performed: filtering each block in the target row from left to right in turn, wherein the filtering progress of the target row lags behind the filtering progress of the last row of the target row by two block filtering time, and the target row is any one of the one or more subsequent rows.

[0092] The following will be illustrated by combining Figure 4 with examples:

[0093] In this example, the first row to the fourth row correspond to different threads respectively.

[0094] If blocks 0-5 are the case of the first row, the filtering of blocks 0-5 is performed from left to right, which is not constrained by other conditions. The second row to the fourth row will be constrained by the filtering time of their last row. As shown in Figure 5 , the numbers in the middle of the quadrilateral block represent the number of blocks, and the numbers in the circle represent the filtering operation order of the block. The filtering order is:

[0095] First filtering operation: block 0;

[0096] Second filtering operation: block 1;

[0097] Third filtering operation (in parallel): block 2, block 6;

[0098] Fourth filtering operation (in parallel): block 3, block 7;

[0099] Fifth filtering operation (in parallel): block 4, block 8, block 12;

[0100] Sixth filtering operation (in parallel): block 5, block 9, block 13;

[0101] Seventh filtering operation (in parallel): block 10, block 14, block 18;

[0102] Eighth filtering operation (in parallel): block 11, block 15, block 19;

[0103] Ninth filtering operation (in parallel): block 16, block 20;

[0104] Tenth filtering operation (in parallel): block 17, block 21;

[0105] Eleventh filtering operation: block 22;

[0106] Twelfth filtering operation: block 23.

[0107] Based on the flow of the embodiment, parallel filtering is adopted in the third to tenth filtering operations. For example, in the third filtering operation, block 2 can be filtered by thread #1 corresponding to the first row, and block 6 can be filtered by thread #2 corresponding to the second row. Since block 2 and block 6 are filtered by different threads, they can be filtered in parallel.

[0108] Therefore, to filter the 24 blocks in Figure 4 , the traditional filtering method needs to perform 24 filtering operations in time sequence, corresponding to 24 filtering time units. However, the technical solution of the embodiment only needs 12 filtering operations (12 filtering time units), which improves the filtering efficiency and reduces the time required for filtering.

[0109] In an optional embodiment, the plurality of rows include an mth row and one or more subsequent rows, m being an integer greater than or equal to 2. The step S302 of performing parallel filtering operation on at least two blocks in the plurality of blocks can include:

[0110] filtering each block in the target row from left to right by the thread assigned to the target row;

[0111] wherein the filtering progress of the target row lags behind the filtering progress of the previous row of the target row by two block filtering times, and the target row is the mth row or any one of the one or more subsequent rows.

[0112] If blocks 0-5 are not the first row, the filtering of blocks 0-5 is performed from left to right and is constrained by the previous row. It can be seen that one or more new threads can be added in the decoding process to further improve the efficiency as needed.

[0113] In an optional embodiment, the target row corresponds to the nth row, n being an integer greater than or equal to 2; accordingly, the step S302 of performing parallel filtering operation on at least two blocks in the plurality of blocks can include: (1) recording the number of filtered blocks in each row, the filtering data identifying the number of blocks that have been filtered; (2) in the case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, filtering the blocks in the nth row that have not been filtered; wherein the maximum preset value is the total number of blocks in the (n-1)th row; (3) in the case where the number of filtered blocks is greater than 2 and less than or equal to the maximum preset value, filtering the corresponding blocks in the nth row, the position of the corresponding blocks in the nth row being the number of filtered blocks minus 2. Through the above recording and setting method, the blocks can be controlled to be filtered at a predetermined progress, ensuring that the blocks do not violate the filtering rules in AV1 due to early filtering, nor are they delayed.

[0114] In an optional embodiment, the method further comprises:

[0115] In the case where the number of filtered blocks in the (n-1)th row is greater than a maximum preset value, the thread of the (n-1)th row is configured to perform filtering operation on the (n+N-1)th row, N being a positive integer and representing the total number of the plurality of threads.

[0116] If the number of rows formed by the plurality of blocks exceeds a preset number. The first way is to configure a preset number of threads, each thread being responsible for filtering a row. This first way requires starting a large number of threads, occupying more computer resources. The second way is to set a plurality of threads (the number of threads is less than the preset number). The second way is to dynamically switch threads to different rows. Taking thread #1 as an example, when blocks 0-5 are filtered, thread #1 needs to switch to the first row (from top to bottom) that has not been allocated to a thread and has not started filtering, thereby saving the number of threads.

[0117] In order to make the present application better and easier to understand, an application example is provided as follows.

[0118] As shown in Figure 6 The numbers in the middle of the quadrilateral blocks represent the block numbers, and the numbers in the brackets represent the block coordinates (y, x).

[0119] F(y, x) = 2y + x + 1; the filtering order of each block can be calculated by the above formula, and the filtering order takes 1 as the initial value.

[0120] For example, the block of coordinate (2, 0) is taken as an example, the filtering order is 2*0+0+1=5. It can be obtained that under the processing manner of the embodiment, the earliest processing order of the coordinate (2, 0) that can be processed is 5, which is earlier than the previous processing order 12.

[0121] The above-mentioned partitioning needs 12 filtering operations to complete filtering, which can be accelerated by 50% compared with the original 24 operations. For a general case: parallel filtering needs 2y+x+1 operations, and row filtering needs (x+1)*(y+1) operations, which can improve the efficiency by (1-(2y+x+1) / ((x+1)*(y+1)))*100%.

[0122] As shown in Figure 7 , the specific flow operation can be as follows:

[0123] Step S700: Start N threads, declare an array lf_block_count[N] for recording the number of blocks that have been filtered, and initialize it to 0. It should be noted that the number of threads can be set as needed.

[0124] Step S702: Each thread processes the filtering operation of a row of blocks, and after processing each block, the corresponding lf_block_count number is increased by one, that is, assuming that the nth thread processes the filtering of the lf_block_count[n]th block in the nth row, if the filtering of the lf_block_count[n]th block is completed, if_block_count[n]++.

[0125] Step S704: Determine whether all blocks in the nth row are completed filtering.

[0126] If yes, go to step S706; otherwise, go to step S706.

[0127] Step S706: Determine that all blocks in the (n+1)th row can be filtered, and are no longer limited by if_block_count[N]-2.

[0128] Step S708: Determine whether the number of blocks if_block_count[n] of the nth row filtering is completed is greater than or equal to 2.

[0129] If yes, go to step S710; otherwise, go to step S702.

[0130] Step S710: Process the filtering of the if_block_count[N]-2th block in the (n+1)th row. Go to step S702.

[0131] In AV1, the filtering operation involves a large number of pixel-level calculations, which consumes a lot of time. Using the multi-thread filtering method, the encoding can be accelerated in live broadcast and on-demand, so the filtering scheme is of great practical value.

[0132] Embodiment Two

[0133] Figure 8 A block diagram of a filtering device of AV1 according to Embodiment Two of the present application is shown schematically. The filtering device of AV1 can be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments capable of completing a specific function, and the functions of the program modules in the embodiments will be described in detail below. As shown in the figure, the filtering device 800 of AV1 can include a determination module 810 and a filtering module 820, wherein: Figure 8 The determination module 810 is configured to determine a to-be-filtered partition, the to-be-filtered partition including a plurality of blocks, the plurality of blocks forming a plurality of rows.

[0134] The filtering module 820 is configured to perform parallel filtering operations on at least two blocks in the plurality of blocks through a plurality of threads, wherein the plurality of threads are responsible for filtering operations of a plurality of rows in a one-to-one correspondence, the plurality of rows including at least two consecutive rows in the plurality of rows, and each block in the at least two blocks is located in a different row.

[0135] In an optional embodiment, the plurality of rows include a first row and one or more subsequent rows; and the filtering module 820 is further configured to:

[0136] for the first row, perform the following filtering operation: sequentially filtering each block of the first row from left to right;

[0137] for the one or more subsequent rows, perform the following filtering operation: sequentially filtering each block in a target row from left to right, wherein a filtering progress of the target row lags behind a filtering progress of a previous row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows.

[0138] In an optional embodiment, the plurality of rows include an mth row and one or more subsequent rows, m being an integer greater than or equal to 2; and the filtering module 820 is further configured to:

[0139] sequentially filtering each block in a target row from left to right by a thread assigned to the target row;

[0140]

[0141] ​The filtering progress of the target row is two block filtering times behind the filtering progress of the previous row of the target row, and the target row is the mth row or any one of the one or more subsequent rows.

[0142] In an optional embodiment, the filtering module 820 is further configured to:

[0143] Record the number of filtered blocks of each row, and the filtering data identifies the number of currently filtered blocks.

[0144] In a case where the number of filtered blocks in the n-1th row is greater than a maximum preset value, filter the currently unfiltered blocks in the nth row, wherein the maximum preset value is the total number of blocks in the n-1th row, and n is an integer greater than or equal to 2.

[0145] In a case where the number of filtered blocks is greater than 2 and less than or equal to the maximum preset value, filter the corresponding blocks in the nth row, and the position of the corresponding blocks in the nth row is the number of filtered blocks minus 2.

[0146] In an optional embodiment, the encoding device further includes a configuration module (not identified) configured to:

[0147] In a case where the number of filtered blocks in the n-1th row is greater than a maximum preset value, configure the thread of the n-1th row to perform filtering operation on the n+N-1th row, and N is a positive integer and is used to represent the total number of the plurality of threads.

[0148] Embodiment Three

[0149] Figure 9 A hardware architecture schematic diagram of a computer device 10000 suitable for implementing the filtering method of AV1 according to Embodiment Three of the present application is schematically shown. The computer device 10000 is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. For example, it can be a smart phone, a tablet computer, a PC, a virtual reality device, etc. As shown in the figure, the computer device 10000 at least includes but is not limited to a memory 10010, a processor 10020, and a network interface 10030 which can be linked in communication through a system bus. Among them: Figure 9

[0150] ​The memory 10010 includes at least one type of computer-readable storage media, such as a flash memory, a hard disk, a multimedia card micro (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 10010 can be an internal memory module of the computer device 10000, such as a hard disk or a memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external memory device of the computer device 10000, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Of course, the memory 10010 can include both an internal memory module and an external memory device of the computer device 10000. In this embodiment, the memory 10010 is generally used to store an operating system and various application programs installed in the computer device 10000, such as program codes of the AV1 filtering method, and the like. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0151] The processor 10020 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication of the computer device 10000, and the like. In this embodiment, the processor 10020 is used to run program codes or process data stored in the memory 10010.

[0152] The network interface 10030 can include a wireless network interface or a wired network interface, which is usually used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 with an external user terminal through a network, establish a data transmission channel and a communication link between the computer device 10000 and the external user terminal, and the like. The network can be an Intranet, the Internet, a Global System for Mobile Communications (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, and the like wireless or wired network.

[0153] It should be noted that, Figure 9 Only the computer device with components 10010-10030 is shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0154] In this embodiment, the filtering method of AV1 stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (processor 10020 in this embodiment) to complete the embodiments of the present application.

[0155] Embodiment Four

[0156] The present application also provides a computer readable storage medium, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the filtering method of AV1 in Embodiment One.

[0157] In this embodiment, the computer readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the computer readable storage medium can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the computer readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer readable storage medium is usually used to store the operating system and various application software installed on the computer device, such as the program code of the AV1 filtering method in the embodiment, etc. In addition, the computer readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0158] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiments of the present application can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and optionally, they can be realized by program codes executable by the computing device, so that they can be stored in the storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into each integrated circuit module, or multiple modules or steps among them can be manufactured into a single integrated circuit module. Therefore, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0159] It should be noted that the above is only the preferred embodiment of the present application, and does not limit the patent protection scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A filtering method of AV1, characterized in that, The method comprises the following steps: determining a to-be-filtered partition, the to-be-filtered partition comprising a plurality of blocks, the plurality of blocks forming a plurality of rows; performing parallel filtering operations on at least two blocks in the plurality of blocks through a plurality of threads; wherein the plurality of threads are responsible for filtering operations on a plurality of rows in a one-to-one correspondence, the plurality of rows comprising at least two consecutive rows in the plurality of rows, and each of the at least two blocks is located in a different row; wherein the plurality of rows comprises a first row and one or more subsequent rows; and the performing of the parallel filtering operations on the at least two blocks in the plurality of blocks comprises: for the first row, performing filtering operations in the order of left to right on each block in the first row; for the one or more subsequent rows, performing filtering operations in the order of left to right on each block in a target row, wherein the filtering progress of the target row lags behind the filtering progress of the last row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows; wherein the target row corresponds to the nth row, n is an integer greater than or equal to 2; and the performing of the parallel filtering operations on the at least two blocks in the plurality of blocks comprises: recording the number of filtered blocks of each row, the number of filtered blocks indicating the number of blocks that have been filtered; in the case that all the filtered blocks in the (n-1)th row have completed filtering, filtering the blocks in the nth row that have not been filtered; in the case that the number of filtered blocks is greater than 2 and less than or equal to a maximum preset value, filtering the corresponding blocks in the nth row, the position of the corresponding blocks in the nth row being the number of filtered blocks minus 2; wherein the maximum preset value is the total number of blocks in the (n-1)th row.

2. The method of claim 1, wherein, The plurality of rows comprises the mth row and one or more subsequent rows, m being an integer greater than or equal to 2; and the performing of the parallel filtering operations on the at least two blocks in the plurality of blocks comprises: filtering each block in a target row in the order of left to right through a thread assigned to the target row; wherein the filtering progress of the target row lags behind the filtering progress of the last row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows or the mth row.

3. The method of claim 1, wherein, The method further comprises: in the case that all the filtered blocks in the (n-1)th row have completed filtering, configuring the thread of the (n-1)th row to perform filtering operations on the (n+N-1)th row, N being a positive integer and representing the total number of the plurality of threads.

4. A filtering apparatus of AV1, characterized by, The method comprises the following steps: a determining module is configured to determine a to-be-filtered partition, the to-be-filtered partition comprising a plurality of blocks, the plurality of blocks forming a plurality of rows; a filtering module is configured to perform parallel filtering operations on at least two blocks in the plurality of blocks through a plurality of threads; wherein the plurality of threads are responsible for filtering operations on a plurality of rows in a one-to-one correspondence, the plurality of rows comprising at least two consecutive rows in the plurality of rows, and each of the at least two blocks is located in a different row. The filter module is further configured to: for the first row, perform a filtering operation of filtering each block of the first row in sequence from left to right; and for the one or more subsequent rows, perform a filtering operation of filtering each block in a target row in sequence from left to right, where a filtering progress of the target row lags behind a filtering progress of a previous row of the target row by two block filtering times, and the target row is any one of the one or more subsequent rows. The target row corresponds to an nth row, where n is an integer greater than or equal to 2; the filter module is further configured to: record a number of filtered blocks of each row, where the number of filtered blocks indicates a number of blocks that have been filtered; in a case where the filtered blocks in the (n-1)th row have all completed filtering, filter a block that has not been filtered in the nth row; in a case where the number of filtered blocks is greater than 2 and less than or equal to a maximum preset value, filter a corresponding block in the nth row, where a position of the corresponding block in the nth row is the number of filtered blocks minus 2; and the maximum preset value is a total number of blocks in the (n-1)th row.

5. The apparatus of claim 4, wherein, The configuration module is further configured to: in a case where the filtered blocks in the (n-1)th row have all completed filtering, configure a thread of the (n-1)th row to perform a filtering operation on an (n+N-1)th row, where N is a positive integer and represents a total number of the plurality of threads.

6. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the computer program, is configured to implement the steps of the AV1 filtering method in any one of claims 1 to 3.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executable by at least one processor to cause the at least one processor to perform the steps of the AV1 filtering method in any one of claims 1 to 3.

8. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by the processor, implement the steps of the AV1 filtering method in any one of claims 1 to 3.

Images