Data processing method, data processing device, electronic equipment and readable medium
By performing vertical and horizontal deblocking filtering of video blocks, visual defects caused by block effect are solved, and the streamlined processing of deblocking filtering is realized, which improves the efficiency and image quality of video encoding.
Patent Information
- Application Number
- CN202311113359.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-30
- Publication Date
- 2025-03-07
AI Technical Summary
In the existing video encoding technology, visual flaws caused by block effect affect the subjective quality of the image, and the deblocking filtering efficiency is low and the resource occupies a large amount.
By performing vertical and horizontal deblocking filtering on the input video block, the vertical filter unit and the horizontal filter unit are determined respectively, and the information of the target video block is output without affecting subsequent filtering, so as to realize the flow-through processing of deblocking filtering.
It reduces the consumption of storage resources, reduces the hardware area, improves hardware performance, and improves the efficiency and image quality of video encoding.
Smart Images
Figure CN119583822A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a data processing method, a data processing device, an electronic device, and a computer-readable medium. Background Art
[0002] With the advent of the Internet era and the development of digital media, video consumption and Internet traffic are gradually increasing, and digital video applications are gradually developing towards high definition, high frame rate, and high compression rate, which puts higher requirements on video encoding and decoding technology. Advanced Video Coding (AVC), or H.264 encoding, has gradually failed to meet the needs of current video application development; High Efficiency Video Coding (HEVC), or H.265 encoding, cannot be well applied in streaming media due to its compatibility, real-time encoding difficulty, authorization and other factors.
[0003] In order to meet the needs of the current development of video applications, encoder technology has been continuously developed, and a variety of new encoders have been produced. For example, the AOM Video 1 (AV1) codec developed by the Alliance for Open Media (AOM). AV1 is a web-based application designed as an open video coding format for video transmission over the Internet. It supports high-quality real-time video delivery, has high scalability, and supports 8k ultra-high definition video (UHD).
[0004] Common encoders such as H.264 / AVC encoder, H.265 / HEVC encoder, and AV1 all process videos based on blocks. During video encoding, the video frame is divided into multiple video blocks for encoding operations, which will cause the adjacent edges of two encoded video blocks to appear visually discontinuous, resulting in visual artifacts in the decoded video frames at the decoding end. This visual artifact is called a block effect. The main reason for the block effect is the artificial boundaries caused by the block-based codec. The block effect is particularly obvious at low bit rates and will greatly affect the subjective quality of the image; moreover, the reconstructed video frame will be used as a reference frame in the encoding process of subsequent video frames. The block effect of the reconstructed video frame will affect the encoding efficiency and quality of subsequent video frames.
[0005] Deblocking filter (DBF), also known as deblocking filter, is a filter that reduces visual artifacts at the boundaries of video blocks and is used to solve the blocking problem in reconstructed video frames. H.264, H.265, and AV1 usually use deblocking filtering technology to improve encoding quality during encoding and decoding, that is, traversing the blocks in the reconstructed image, smoothing the boundaries of each traversed block to reduce the blocking effect at the block boundary, thereby improving the subjective quality of the image.
[0006] However, a large amount of resources are required when performing deblocking filtering, and the efficiency of deblocking filtering is low. Summary of the invention
[0007] The embodiments of the present disclosure provide a data processing method, a data processing device, an electronic device, and a computer-readable medium.
[0008] In a first aspect, an embodiment of the present disclosure provides a data processing method, comprising: determining a vertical filtering unit according to an input video block, wherein a first influence area of vertical deblocking filtering is used to determine the vertical filtering unit; performing vertical deblocking filtering on the vertical filtering unit; determining a horizontal filtering unit according to the input video block, wherein the first influence area and a second influence area of horizontal deblocking filtering are used to determine the horizontal filtering unit; performing horizontal deblocking filtering on the horizontal filtering unit; and outputting information of a target video block corresponding to the input video block.
[0009] In a second aspect, an embodiment of the present disclosure provides a data processing device, comprising a processing module, a deblocking filtering module, and an output module; the processing module is used to determine a vertical filtering unit according to an input video block, wherein a first influence area of the vertical deblocking filter is used to determine the vertical filtering unit; the deblocking filtering module is used to perform vertical deblocking filtering on the vertical filtering unit; the processing module is also used to determine a horizontal filtering unit according to the input video block, wherein the first influence area and a second influence area of the horizontal deblocking filter are used to determine the horizontal filtering unit; the deblocking filtering module is also used to perform horizontal deblocking filtering on the horizontal filtering unit; the output module is used to output information of a target video block corresponding to the input video block.
[0010] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method described in the first aspect of the embodiment of the present disclosure.
[0011] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the data processing method described in the first aspect of the embodiment of the present disclosure is implemented.
[0012] In the data processing method, data processing device, and electronic device provided by the embodiments of the present disclosure, each time a video block is input, vertical deblocking filtering is first performed on the vertical filtering unit, and then horizontal deblocking filtering is performed on the horizontal filtering unit that will no longer be affected by the deblocking filtering, and the information of the video block that is no longer affected by the vertical deblocking filtering and no longer affected by the horizontal deblocking filtering is output. In the process of sequentially inputting the video blocks of the input image frame, pipeline processing of deblocking filtering is realized; since the information of the target video block can be output simultaneously during the deblocking filtering process, it is not necessary to store the intermediate information of all the blocks of the image frame, which reduces the consumption of storage resources, is conducive to the hardware implementation of deblocking filtering, and can also reduce the hardware area and improve the hardware performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 is a flow chart of a data processing method in an embodiment of the present disclosure;
[0014] Figure 2 is a flowchart of some steps in another data processing method in an embodiment of the present disclosure;
[0015] Figure 3 is a schematic diagram of a filtering unit in an embodiment of the present disclosure;
[0016] Figure 4 is a schematic diagram of a buffer zone in an embodiment of the present disclosure;
[0017] Figure 5 is a schematic diagram of another buffer zone in an embodiment of the present disclosure;
[0018] Figure 6 is a schematic diagram of the composition of a data processing device in an embodiment of the present disclosure;
[0019] Figure 7 is a schematic diagram of the composition of an electronic device in an embodiment of the present disclosure;
[0020] Figure 8 It is a schematic diagram of the composition of another electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the data processing method, data processing device, electronic device, and computer-readable medium provided by the present disclosure are described in detail below in conjunction with the accompanying drawings.
[0022] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.
[0023] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.
[0024] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0025] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.
[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.
[0027] AV1 divides frames into super blocks (SB) and transform blocks (TB), and evaluates all boundaries from 64×64 blocks to 4×4 blocks. Compared with HEVC DBF, which processes block boundaries of 8×8 blocks, AV1 DBF is more complex than HEVC DBF in terms of filtering level, number of filters used, number of filter taps, and filtering boundaries. For HEVC DBF, some hardware solutions can process 8K UHD video and achieve a throughput of 60fps. However, the hardware solution of HEVC DBF is not applicable to AV1 DBF. The AV1 protocol stipulates that video images use 4×4 pixel blocks as basic blocks, and DBF is applied to the boundaries of super blocks and transform blocks. When deblocking filtering, the vertical boundaries of the entire frame are first deblocked, and then the horizontal boundaries of the entire frame are deblocked. In some related technologies, some specific filtering decisions are used to define whether to apply a deblocking filter on the boundary of each transform block. In some related technologies, deblocking filtering will modify up to 6 sample values on both sides of the current filter boundary, that is, information of 6 pixels, and will affect 2 4×4 blocks on both sides of the current filter boundary. When the whole frame image is first deblocked vertically and then deblocked horizontally, a lot of intermediate data needs to be stored, which is difficult to implement in hardware; the value after each filtering may be changed in the next filtering, resulting in the need to save the intermediate temporary data multiple times, making the deblocking filtering inefficient; the whole frame image is first deblocked vertically and then deblocked horizontally before the information is output. The subsequent processing modules of the deblocking filter need to wait for the DBF processing to be completed, which seriously affects the efficiency of image data processing.
[0028] In view of this, firstly, refer to Figure 1 , the present disclosure provides a data processing method, including:
[0029] S1. Determine a vertical filtering unit according to an input video block, wherein a first influence area of vertical deblocking filtering is used to determine the vertical filtering unit;
[0030] S2, performing vertical deblocking filtering on the vertical filtering unit;
[0031] S3, determining a horizontal filtering unit according to the input video block, wherein the first influence area and the second influence area of the horizontal deblocking filter are used to determine the horizontal filtering unit;
[0032] S4, performing horizontal deblocking filtering on the horizontal filtering unit;
[0033] S5. Output information of a target video block corresponding to the input video block.
[0034] In the disclosed embodiment, the input image frame is divided into video blocks, and the video blocks of the input image frame are sequentially used as input video blocks, wherein the input video blocks are blocks obtained after intra / inter prediction, transformation, quantization, and entropy coding.
[0035] The embodiment of the present disclosure does not impose any special limitation on the size of the video block. For example, the video block may be a 4×4 block, an 8×8 block, a 64×64 block, etc.
[0036] In the disclosed embodiment, multiple sample values on both sides of the filter boundary, that is, information of multiple pixels, will be modified during deblocking filtering, so multiple video blocks on both sides of the filter boundary will be affected. In the disclosed embodiment, the affected area refers to the area composed of video blocks that will be affected by deblocking filtering. For example, some deblocking filters will modify up to 6 sample values on the left and right sides of the current filter boundary, that is, information of 6 pixels, which will affect 2 4×4 blocks on the left and right sides of the current filter boundary. In the disclosed embodiment, the first affected area represents the video block affected by the vertical deblocking filter, and the second affected area represents the video block affected by the horizontal deblocking filter. In the disclosed embodiment, the first affected area and the second affected area are related to the size of the video block and the number of sample values that will be modified by the deblocking filter. The disclosed embodiment does not make any special limitation on this.
[0037] In the embodiment of the present disclosure, the vertical filtering unit is determined by the first influence area, which means that the vertical filtering unit is composed of video blocks that will be affected by the vertical deblocking filter. In the embodiment of the present disclosure, the horizontal filtering unit is determined by the first influence area and the second influence area, which means that the horizontal filtering unit is composed of video blocks that will be affected by the horizontal deblocking filter but will not be affected by the vertical deblocking filter. For example, the video blocks that constitute the horizontal filtering unit are located outside the first influence area and will not be affected by the vertical deblocking filter in the subsequent deblocking filtering process.
[0038] In the disclosed embodiment, the target video block refers to a video block that will not be affected by the vertical deblocking filter or the horizontal deblocking filter in the subsequent deblocking filtering process. For example, the target video block is a video block located outside the first influence area and outside the second influence area. When the video block is no longer affected by the vertical deblocking filter or the horizontal deblocking filter, the block information of the video block will no longer be modified during the entire deblocking filtering process of the input image frame, and the information of the video block can be output, for example, transmitted to a subsequent processing module.
[0039] In the disclosed embodiment, each time a video block is input, the vertical filtering unit is first subjected to vertical deblocking filtering through steps S1 to S4, and then the horizontal filtering unit that will no longer be affected by the deblocking filtering is subjected to horizontal deblocking filtering, and the information of the video block that is no longer affected by the vertical deblocking filtering and the horizontal deblocking filtering is output. In the process of sequentially inputting the video blocks of the input image frame, the pipeline processing of the deblocking filtering is realized; since the information of the target video block can be output simultaneously during the deblocking filtering process, it is not necessary to store the intermediate information of all the blocks of the image frame, which reduces the consumption of storage resources, is conducive to the hardware to realize the deblocking filtering, reduces the hardware area, and improves the hardware performance.
[0040] In the embodiment of the present disclosure, the first affected area is related to the size of the video block and the number of sample values that will be modified by the deblocking filter.
[0041] Accordingly, in some embodiments, reference Figure 2 , determining a vertical filtering unit according to an input video block, including:
[0042] S11, determining a video block to be vertically deblocked according to the input video block;
[0043] S12: Determine the vertical filtering unit according to the position of the video block to be vertically deblocked and the first affected area.
[0044] The disclosed embodiment does not specifically limit how to determine the video block to be vertically deblocked according to the input video block.
[0045] In some embodiments, determining a video block to be vertically deblocked filtered according to the input video block comprises:
[0046] The video block to be vertically deblocked is determined according to the following formula:
[0047] filter_position_ver(row)=block_position(row) (1)
[0048] filter_position_ver(col)=block_position(col) -n+1 (2)
[0049]
[0050] Wherein, filter_position_ver(row) represents the row number of the video block to be vertically deblocked, filter_position_ver(col) represents the column number of the video block to be vertically deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
[0051] In some embodiments, performing vertical deblocking filtering on the vertical filtering unit includes:
[0052] Perform vertical deblocking filtering on the left boundary of the video block to be vertically deblocked.
[0053] In the embodiment of the present disclosure, the second affected area is related to the size of the video block and the number of sample values that will be modified by the deblocking filter.
[0054] Accordingly, in some embodiments, reference Figure 2 , determining a horizontal filtering unit according to the input video block, comprising:
[0055] S31, determining a video block to be horizontally deblocked according to the input video block;
[0056] S32: Determine the horizontal filtering unit according to the position of the video block to be horizontally deblocked and the second affected area.
[0057] In some embodiments, determining a video block to be horizontally deblocked according to the input video block comprises:
[0058] The video block to be horizontally deblocked is determined according to the following formula:
[0059] filter_position_hor(row)=block_position(row) -n+1 (3)
[0060] filter_position_hor(col)=block_position(col)-n-1 (4)
[0061]
[0062] Wherein, filter_position_hor(row) represents the row number of the video block to be horizontally deblocked, filter_position_hor(col) represents the column number of the video block to be horizontally deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
[0063] In some embodiments, performing horizontal deblocking filtering on the horizontal filtering unit includes:
[0064] Perform horizontal deblocking filtering on the upper boundary of the video block to be horizontally deblocking filtered.
[0065] In the disclosed embodiment, the vertical filtering unit is determined by formula (1) and formula (2), and then the horizontal filtering unit is determined by formula (3) and formula (4), which can ensure that the video blocks constituting the horizontal filtering unit will not be affected by the vertical deblocking filtering in the subsequent deblocking filtering process.
[0066] For example, taking a video block of 4×4 and deblocking filtering affecting 6 pixels on both sides of the filtering boundary (i.e., 2 4×4 blocks), formula (1) and formula (2) can be expressed as:
[0067] filter_position_ver(row)=block_position(row);
[0068] filter_position_ver(col)=block_position(col)-1;
[0069] Formula (3) and formula (4) can be expressed as:
[0070] filter_position_hor(row)=block_position(row)-1;
[0071] filter_position_hor(col)=block_position(col)-3;
[0072] like Figure 3As shown in (a), the size of the image frame is 16×16 pixels, 0-15 are 4×4 blocks, and they are input one by one from left to right and from top to bottom. The block represented by 15 is the currently received block, that is, the input video block. According to the above formula, the video block to be vertically deblocked is 14 blocks, and the filtering boundary is the left boundary of 14 blocks. The pixel values in 12 blocks and 13 blocks on the left side of the filtering boundary and 14 blocks and 15 blocks on the right side will be modified at most. Figure 3 As shown in (b), 15b lock, 12b lock, 13b lock, and 14b lock form a vertical filtering unit. After the left boundary of 14b lock is vertically deblocked, the pixel values in 12b lock will not be modified when the left boundary of 15b lock is subsequently vertically deblocked. According to the above formulas (3) and (4), 8b lock is determined as the video block to be horizontally deblocked, and the filtering boundary is the upper boundary of 8b lock. The pixel values in 0b lock and 4b lock above the filtering boundary and in 8b lock and 12block below the filtering boundary will be modified at most. Figure 3 As shown in (c), 8b lock, 0b lock, 4b lock, and 12b lock form a horizontal filtering unit. Figure 3 As shown, according to the above formula, in Figure 3 After the horizontal filtering unit shown in (c) performs horizontal filtering, 0block will no longer be used as a video block constituting a vertical filtering unit, nor will it be used as a video block constituting a horizontal filtering unit, so the information of 0block lock can be output.
[0073] The disclosed embodiments do not specifically limit how to store the intermediate information of the video blocks in the deblocking filtering process.
[0074] In some embodiments, a first buffer and a second buffer are set, the first buffer is used to store intermediate information required for vertical deblocking filtering, and the second buffer is used to store intermediate information required for horizontal deblocking filtering.
[0075] In some embodiments, vertical deblocking filtering is first performed on a vertical filtering unit, and then horizontal deblocking filtering is performed on a horizontal filtering unit that will no longer be affected by deblocking filtering. The first buffer stores intermediate information of video blocks that will subsequently be affected by vertical deblocking filtering, and the second buffer stores intermediate information of video blocks that will subsequently not be affected by vertical deblocking filtering.
[0076] Accordingly, in some embodiments, performing vertical deblocking filtering on the vertical filtering unit includes:
[0077] Acquire first intermediate information of a plurality of video blocks constituting the vertical filtering unit from a first buffer;
[0078] Performing vertical deblocking filtering on the vertical filtering unit according to the first intermediate information to obtain second intermediate information of a plurality of video blocks constituting the vertical filtering unit;
[0079] Updating the second intermediate information into the first buffer;
[0080] The intermediate information of the video blocks outside the first impact area in the first buffer is updated to the second buffer.
[0081] It should be noted that in the disclosed embodiment, for each input video block, when it is determined that the input video block needs to be subjected to vertical deblocking filtering, vertical deblocking filtering will be performed once. For each input video block, before the vertical deblocking filtering is performed, the information of the video block that will be affected by the vertical deblocking filtering stored in the first buffer is the first intermediate information; the information of the video block that is affected by the vertical deblocking filtering obtained after the vertical deblocking filtering is performed is the second intermediate information. For example, in combination with Figure 4 , the numbers represent video blocks, and the superscripts of the numbers represent the number of times the video block has been modified by filtering; assuming that video block 0 has been modified once, the information 0 stored in the first buffer 1 is the first intermediate information; after performing vertical deblocking filtering again, the information obtained is 0 2 It is the second intermediate information.
[0082] The disclosed embodiment does not specifically limit how to perform vertical deblocking filtering on the vertical filtering unit. For example, the position of the video block that needs to be vertically deblocked is calculated, and the data of the adjacent video blocks are taken according to the position to form the corresponding vertical filtering unit, and then whether to perform filtering is analyzed according to the syntax elements sent from the coding segment, and which filter to use for filtering is selected according to the level of the corresponding filtering decision.
[0083] In some embodiments, updating the second intermediate information into the first buffer includes:
[0084] The second intermediate information is updated into the first buffer according to a first-in-first-out order.
[0085] In some embodiments, after performing vertical deblocking filtering on the vertical filtering unit, the method further includes:
[0086] In the case where the input video block is a video block located at the end of a row, inputting at least one idle block in sequence;
[0087] sequentially taking the plurality of idle blocks as new input video blocks, and determining a new vertical filtering unit according to a new output video block;
[0088] Perform vertical deblocking filtering on the new vertical filtering unit.
[0089] For example, Figure 3 The image frame shown is of a size of 16×16 pixels. When the input video block is 3b lock, 3block is a video block at the end of a row. Then, three idle blocks are continuously input. The video block to be vertically deblocked and the filtering boundary are recalculated according to formula (1) and formula (2), and vertical deblocking filtering is performed, and the intermediate value in the first buffer is updated. Among them, when the filtering boundary is the boundary of the idle block, vertical deblocking filtering is not performed, but the first buffer is updated according to the first-in-first-out order, so that the intermediate information in the first buffer is output. When the input video block is 7b lock or 11b lock, the same processing is performed.
[0090] In some embodiments, updating the intermediate information of the video blocks outside the first affected area in the first buffer into the second buffer includes:
[0091] The intermediate information of the video block output from the first buffer is updated to the second buffer according to a first-in-first-out order.
[0092] In some embodiments, performing horizontal deblocking filtering on the horizontal filtering unit includes:
[0093] Acquire third intermediate information of a plurality of video blocks constituting the horizontal filtering unit from the second buffer;
[0094] Performing horizontal deblocking filtering on the horizontal filtering unit according to the third intermediate information to obtain fourth intermediate information of a plurality of video blocks constituting the horizontal filtering unit;
[0095] The fourth intermediate information is updated into the second buffer.
[0096] It should be noted that, in the disclosed embodiment, for each input video block, when it is determined that the input video block needs to be subjected to horizontal deblocking filtering, horizontal deblocking filtering will be performed once. For each input video block, before the horizontal deblocking filtering is performed, the information of the video block that will be affected by the horizontal deblocking filtering stored in the first buffer is the third intermediate information; the information of the video block that will be affected by the vertical deblocking filtering obtained after the vertical deblocking filtering is performed is the fourth intermediate information. For example, in combination with Figure 5 , the numbers represent video blocks, and the superscripts of the numbers represent the number of times the video block has been modified by filtering; assuming that video block 0 has been modified 2 times, the information 0 stored in the second buffer2 is the third intermediate information; after performing vertical deblocking filtering again, the information obtained is 0 3 This is the fourth intermediate information.
[0097] The disclosed embodiment does not specifically limit how to perform horizontal deblocking filtering on the horizontal filtering unit. For example, the position of the video block that needs to be horizontally deblocked is calculated, and the data of the adjacent video blocks are taken according to the position to form the corresponding horizontal filtering unit, and then whether to perform filtering is analyzed according to the syntax elements sent from the coding segment, and which filter to use for filtering is selected according to the level of the corresponding filtering decision.
[0098] In some embodiments, updating the fourth intermediate information into the second buffer includes:
[0099] The fourth intermediate information is updated into the second buffer according to a first-in-first-out order.
[0100] In some embodiments, after performing horizontal deblocking filtering on the horizontal filtering unit, the method further includes:
[0101] In the case where the input video block is a video block located at the end, inputting a plurality of idle blocks in sequence;
[0102] sequentially taking the plurality of idle blocks as new input video blocks, and determining a new horizontal filtering unit according to the new output video block;
[0103] Perform horizontal deblocking filtering on the new horizontal filtering unit.
[0104] For example, Figure 3 The image frame shown is of a size of 16×16 pixels. When the input video block is 15block, 15block is the video block at the end of the row. Then, three idle blocks are continuously input, and the video blocks to be vertically deblocked and the filtering boundary are recalculated according to formula (1) and formula (2), and vertical deblocking filtering is performed, and the intermediate value in the first buffer is updated. When the filtering boundary is the boundary of the idle block, vertical deblocking filtering is not performed, but the first buffer is updated according to the first-in-first-out order, so that the intermediate information in the first buffer is output. Then, seven idle blocks are continuously input as the next row. In the process of inputting the idle blocks, the video blocks to be horizontally deblocked and the filtering boundary are recalculated according to formula (3) and formula (4), and vertical deblocking filtering is performed. When the filtering boundary is the boundary of the idle block, horizontal deblocking filtering is not performed, but the second buffer is updated according to the first-in-first-out order, so that the intermediate information in the second buffer is output.
[0105] In the embodiment of the present disclosure, a first buffer is set to store intermediate information required for vertical deblocking filtering, and a second buffer is set to store intermediate information required for horizontal deblocking filtering, so that vertical deblocking filtering is separated from horizontal deblocking filtering; an idle block is input at the end of a line and an end of an image frame, so as to ensure that vertical deblocking filtering and horizontal deblocking filtering are performed on the boundary of each video block, and data in the first buffer is output to the second buffer according to a first-in-first-out order, and data in the second buffer is output according to a first-in-first-out order.
[0106] In some embodiments, outputting information of a target video block corresponding to the input video block includes:
[0107] The intermediate information output from the second buffer is output as information of the target video block.
[0108] For example, taking the video block as a 4×4 block and the deblocking filter affecting 6 pixels on both sides of the filter boundary (ie, 2 4×4 blocks), the first buffer (Buffer 1) and the second buffer (Buffer 2) are as follows: Figure 4 and Figure 5 As shown, Figure 4 and Figure 5 The numbers in the squares represent the corresponding Figure 3 In the video block (a), the superscript of the number indicates the number of times the video block has been modified by filtering. Buffer 1 stores the intermediate information of the first three blocks for vertical deblocking filtering for each row, and Buffer 2 stores the intermediate information of the first three blocks for horizontal deblocking filtering for each column. The update order is reverse order update, Buffer 1 is updated first, and the blocks updated in reverse order of Buffer 1 are updated into Buffer 2 in the order of reverse update. Reverse order update means updating in the order of first in, first out.
[0109] by Figure 3Take the 0b lock, 1b lock, 2b lock, and 3b lock in the first row of (a) as an example. Considering that the input video block is located at the boundary of the image frame, the 0, 1, 2, 3, and p on the left side of Buffer 1 represent the current input video block 0block, 1b lock, 2b lock, 3b lock, and pb lock, respectively, where pb lock is a free block. When 0b lock is received, no deblocking filtering is performed, and it is updated in reverse order to the last b lock position of Buffer 1; when 1b lock is received, no filtering operation is performed, and Buffer 1 is updated in reverse order; when 2b lock is received, vertical deblocking filtering is performed on the left boundary of 1b lock, and the filtered intermediate information is updated in reverse order to Buffer 1; when 3b lock is received, vertical filtering is performed on the left boundary of 2b lock, and the filtered intermediate information is updated in reverse order to Buffer 1. At this time, 0 2 Update from Buffer 1 to Buffer 2 in reverse order, corresponding to 0 2 The last block lock position in the block lock column; at this time, 3 block lock is already at the end of the row. In order to update all the block locks in Buffer 1 into Buffer 2, three pb locks are received, and Buffer 1 and Buffer 2 are updated three times. Finally, Buffer 2 is as follows Figure 4 The remaining rows are updated similarly.
[0110] Considering the block division method of AV1, Buffer 1 can be expanded to 4 block locks. According to the above buffer update method, when receiving 15 block locks, the status of Buffer 1 and Buffer 2 is as follows: Figure 5 The form shown.
[0111] According to the update order of Buffer 1 and Buffer 2, the first three lines of Buffer 1 can be released by receiving three pb locks. At this time, all the filtered intermediate information is updated into Buffer 2. The vertical filtering unit is composed of 15b lock and the three block locks of Buffer 1 in the current line of 15b lock; the horizontal filtering unit is composed of the first block lock (i.e., 12) that has been modified by the vertical deblocking filter. 2 b lock) corresponds to 12 in Buffer2 2 After the filtering of the horizontal filtering unit is completed, the Buffer is updated in the manner described above.
[0112] In order to achieve consistency of buffer update and completely release buffer space when processing a complete frame image, pb lock can be received three times for each column of block lock, and a buffer update is performed after each reception to output the data of the target video block. Each block lock output can also output the position information corresponding to the block lock, so as to facilitate the subsequent filtering processing of the DBF subsequent module.
[0113] Second, refer to Figure 6 , the embodiment of the present disclosure provides a data processing device, including a processing module 110, a deblocking filter module 120, and an output module 130;
[0114] The processing module 110 is used to determine a vertical filtering unit according to an input video block, wherein a first influence area of vertical deblocking filtering is used to determine the vertical filtering unit;
[0115] The deblocking filter module 120 is used to perform vertical deblocking filtering on the vertical filtering unit;
[0116] The processing module 110 is further configured to determine a horizontal filtering unit according to the input video block, wherein the first influence area and the second influence area of the horizontal deblocking filter are used to determine the horizontal filtering unit;
[0117] The deblocking filter module 120 is also used to perform horizontal deblocking filtering on the horizontal filtering unit;
[0118] The output module 130 is configured to output information of a target video block corresponding to the input video block.
[0119] In the disclosed embodiment, each time a video block is input, vertical deblocking filtering is first performed on the vertical filtering unit, and then horizontal deblocking filtering is performed on the horizontal filtering unit that will no longer be affected by the deblocking filtering, and the information of the video block that is no longer affected by the vertical deblocking filtering and the horizontal deblocking filtering is output. In the process of sequentially inputting the video blocks of the input image frame, pipeline processing of deblocking filtering is realized; since the information of the target video block can be output simultaneously during the deblocking filtering process, it is not necessary to store the intermediate information of all the blocks of the image frame, which reduces the consumption of storage resources, is conducive to the hardware to realize deblocking filtering, and can also reduce the hardware area and improve the hardware performance.
[0120] In some embodiments, the processing module is used to determine a video block to be vertically deblocked according to the input video block; and determine the vertical filtering unit according to a position of the video block to be vertically deblocked and the first affected area.
[0121] In some embodiments, the processing module determines the video block to be vertically deblocked according to the following formula:
[0122] filter_position_ver(row)=block_position(row);
[0123] filter_position_ver(col)=block_position(col)-n+1;
[0124]
[0125] Wherein, filter_position_ver(row) represents the row number of the video block to be vertically deblocked, filter_position_ver(col) represents the column number of the video block to be vertically deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
[0126] In some embodiments, the deblocking filter module includes a vertical deblocking filter submodule;
[0127] The vertical deblocking filtering submodule is used to perform vertical deblocking filtering on the left boundary of the video block to be vertically deblocked.
[0128] In some embodiments, the processing module determines a video block to be horizontally deblocked according to the input video block; and determines the horizontal filtering unit according to a position of the video block to be horizontally deblocked and the second affected area.
[0129] In some embodiments, the processing module determines the video block to be horizontally deblocked according to the following formula:
[0130] filter_position_hor(row)=block_position(row)-n+1;
[0131] filter_position_hor(col)=block_position(col)-n-1;
[0132]
[0133] Wherein, filter_position_hor(row) represents the row number of the video block to be horizontally deblocked, filter_position_hor(col) represents the column number of the video block to be horizontally deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
[0134] In some embodiments, the deblocking filter module includes a deblocking filter submodule;
[0135] The deblocking filter submodule is used to perform horizontal deblocking filtering on the upper boundary of the video block to be horizontally deblocked.
[0136] In some embodiments, the data processing device further comprises a first buffer and a second buffer, and the deblocking filter module comprises a vertical deblocking filter submodule;
[0137] The vertical deblocking filtering submodule is used to obtain first intermediate information of multiple video blocks constituting the vertical filtering unit from the first buffer; perform vertical deblocking filtering on the vertical filtering unit according to the first intermediate information to obtain second intermediate information of the multiple video blocks constituting the vertical filtering unit; update the second intermediate information to the first buffer; and update the intermediate information of the video blocks outside the first influence area in the first buffer to the second buffer.
[0138] In some embodiments, the vertical deblocking filter submodule is used to update the second intermediate information into the first buffer according to a first-in-first-out order.
[0139] In some embodiments, the processing module is further used to sequentially input a plurality of idle blocks when the input video block is a video block at the end of a row; sequentially use the plurality of idle blocks as new input video blocks, and determine a new vertical filtering unit according to the new output video block;
[0140] The vertical deblocking filtering submodule performs vertical deblocking filtering on the new vertical filtering unit.
[0141] In some embodiments, the vertical deblocking filter submodule is used to update the intermediate information of the video block output from the first buffer into the second buffer according to a first-in-first-out order.
[0142] In some embodiments, the deblocking filter module includes a horizontal deblocking filter submodule;
[0143] The horizontal deblocking filtering submodule is used to obtain third intermediate information of multiple video blocks constituting the horizontal filtering unit from the second buffer; perform horizontal deblocking filtering on the horizontal filtering unit according to the third intermediate information to obtain fourth intermediate information of the multiple video blocks constituting the horizontal filtering unit; and update the fourth intermediate information to the second buffer.
[0144] In some embodiments, the horizontal deblocking filter submodule is used to update the fourth intermediate information into the second buffer according to a first-in-first-out order.
[0145] In some embodiments, the processing module is further used to sequentially input a plurality of idle blocks when the input video block is a video block at the end; sequentially use the plurality of idle blocks as new input video blocks, and determine a new horizontal filtering unit according to the new output video block;
[0146] The horizontal deblocking filtering submodule is further used to perform horizontal deblocking filtering on the new horizontal filtering unit.
[0147] In the disclosed embodiment, the vertical deblocking filter submodule and the horizontal deblocking filter submodule are set separately, which can realize parallel execution of vertical deblocking filter and horizontal deblocking filter, which is beneficial to improve the efficiency of deblocking filter and improve hardware performance.
[0148] In some embodiments, the output unit is used to output the intermediate information output from the second buffer as information of the target video block.
[0149] Thirdly, refer to Figure 7 An embodiment of the present disclosure provides an electronic device, including the data processing device described in the second aspect of the embodiment of the present disclosure.
[0150] Fourthly, refer to Figure 8 , an embodiment of the present disclosure provides an electronic device, comprising:
[0151] One or more processors 201;
[0152] A memory 202 on which one or more programs are stored. When the one or more programs are executed by one or more processors, the one or more processors implement the data processing method described in the first aspect of the embodiment of the present disclosure;
[0153] One or more I / O interfaces 203 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.
[0154] Among them, the processor 201 is a device with data processing capabilities, including but not limited to a central processing unit (CPU) and the like; the memory 202 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 203 is connected between the processor 201 and the memory 202, and can realize information interaction between the processor 201 and the memory 202, including but not limited to a data bus (Bus) and the like.
[0155] In some embodiments, the processor 201 , the memory 202 , and the I / O interface 203 are connected to each other via a bus 204 , and further connected to other components of the computing device.
[0156] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the data processing method described in the first aspect of the embodiment of the present disclosure is implemented.
[0157] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0158] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.
Claims
1. A data processing method, comprising: Determine a vertical filtering unit according to an input video block, wherein a first influence area of the vertical deblocking filter is used to determine the vertical filtering unit; Performing vertical deblocking filtering on the vertical filtering unit; Determine a horizontal filtering unit according to the input video block, wherein the first influence area and the second influence area of the horizontal deblocking filter are used to determine the horizontal filtering unit; Performing horizontal deblocking filtering on the horizontal filtering unit; Output information of a target video block corresponding to the input video block.
2. The data processing method according to claim 1, wherein: Determining a vertical filtering unit according to an input video block includes: Determine a video block to be vertically deblocked according to the input video block; The vertical filtering unit is determined according to the position of the video block to be vertically deblocked and the first affected area.
3. The data processing method according to claim 2, wherein: Determining a video block to be vertically deblocked according to the input video block includes: The video block to be vertically deblocked is determined according to the following formula: filter_position_ver(row)=block_position(row); filter_position_ver(col)=block_position(col)-n+1; Wherein, filter_position_ver(row) represents the row number of the video block to be vertically deblocked, filter_position_ver(col) represents the column number of the video block to be vertically deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
4. The data processing method according to claim 3, wherein: Performing vertical deblocking filtering on the vertical filtering unit includes: Perform vertical deblocking filtering on the left boundary of the video block to be vertically deblocked.
5. The data processing method according to claim 1, wherein: Determining a horizontal filtering unit according to the input video block includes: Determine a video block to be horizontally deblocked according to the input video block; The horizontal filtering unit is determined according to the position of the video block to be horizontally deblocked and the second affected area.
6. The data processing method according to claim 5, wherein: Determining a video block to be horizontally deblocked according to the input video block includes: The video block to be horizontally deblocked is determined according to the following formula: filter_position_hor(row)=block_position(row)-n+1; filter_position_hor(col)=block_position(col)-n-1; Wherein, filter_position_hor(row) represents the row number of the video block to be horizontally deblocked, filter_position_hor(col) represents the column number of the video block to be horizontally deblocked, block_position(row) represents the row number of the input video block, block_position(col) represents the column number of the input video block, 2l is the pixel width of the affected area of the deblocking filter, m is the pixel width of the video block, Indicates rounding up.
7. The data processing method according to claim 6, wherein: Performing horizontal deblocking filtering on the horizontal filtering unit includes: Perform horizontal deblocking filtering on the upper boundary of the video block to be horizontally deblocking filtered.
8. The data processing method according to any one of claims 1 to 7, wherein: Performing vertical deblocking filtering on the vertical filtering unit includes: Acquire first intermediate information of a plurality of video blocks constituting the vertical filtering unit from a first buffer; Performing vertical deblocking filtering on the vertical filtering unit according to the first intermediate information to obtain second intermediate information of a plurality of video blocks constituting the vertical filtering unit; Updating the second intermediate information into the first buffer; The intermediate information of the video blocks outside the first impact area in the first buffer is updated to the second buffer.
9. The data processing method according to claim 8, wherein: Updating the second intermediate information into the first buffer includes: The second intermediate information is updated into the first buffer according to a first-in-first-out order.
10. The data processing method according to claim 9, wherein: After performing vertical deblocking filtering on the vertical filtering unit, the method further includes: In the case where the input video block is a video block located at the end of a row, inputting at least one idle block in sequence; sequentially taking the plurality of idle blocks as new input video blocks, and determining a new vertical filtering unit according to a new output video block; Perform vertical deblocking filtering on the new vertical filtering unit.
11. The data processing method according to claim 9, wherein: Updating the intermediate information of the video blocks outside the first affected area in the first buffer into the second buffer includes: The intermediate information of the video block output from the first buffer is updated to the second buffer according to a first-in-first-out order.
12. The data processing method according to claim 8, wherein: Performing horizontal deblocking filtering on the horizontal filtering unit includes: Acquire third intermediate information of a plurality of video blocks constituting the horizontal filtering unit from the second buffer; Performing horizontal deblocking filtering on the horizontal filtering unit according to the third intermediate information to obtain fourth intermediate information of a plurality of video blocks constituting the horizontal filtering unit; The fourth intermediate information is updated into the second buffer.
13. The data processing method according to claim 12, wherein: Updating the fourth intermediate information into the second buffer includes: The fourth intermediate information is updated into the second buffer according to a first-in-first-out order.
14. The data processing method according to claim 13, wherein: After performing horizontal deblocking filtering on the horizontal filtering unit, the method further comprises: In a case where the input video block is a video block located at the end, inputting at least one idle block in sequence; sequentially taking the plurality of idle blocks as new input video blocks, and determining a new horizontal filtering unit according to the new output video block; Perform horizontal deblocking filtering on the new horizontal filtering unit.
15. The data processing method according to claim 13, wherein: Outputting information of a target video block corresponding to the input video block includes: The intermediate information output from the second buffer is output as information of the target video block.
16. A data processing device, comprising a processing module, a deblocking filter module, and an output module; The processing module is used to determine a vertical filtering unit according to an input video block, wherein: The first influence area of the vertical deblocking filter is used to determine the vertical filtering unit; The deblocking filter module is used to perform vertical deblocking filtering on the vertical filtering unit; The processing module is further used to determine a horizontal filtering unit according to the input video block, wherein the first influence area and the second influence area of the horizontal deblocking filter are used to determine the horizontal filtering unit; The deblocking filtering module is also used to perform horizontal deblocking filtering on the horizontal filtering unit; The output module is used to output information of a target video block corresponding to the input video block.
17. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method according to any one of claims 1 to 15.
18. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the data processing method according to any one of claims 1 to 15 is implemented.