Method and apparatus for implementing parallel video encoding

Abstract

The application discloses a kind of implementation methods of video parallel encoding. The video frame is divided using the tile TILE and / or SLICE defined in H.264 or H.265 standard, and the minimum in the obtained TILE or SLICE is called subframe. When carrying out intra-frame parallel encoding, each subframe is independently encoded respectively;When all subframes are encoded, the current frame is encoded. When carrying out inter-frame parallel encoding, the integer pixel motion estimation of the encoding block (m, n) of the current frame is carried out after waiting for the encoding block row where the encoding block (m, m+k) of all reference frames is located to be encoded, and the sub-pixel motion estimation of the encoding block (m, n) of the current frame is carried out after waiting for the encoding block row where the encoding block (m, m+k+N) of all reference frames is located to be encoded. The application realizes intra-frame parallel and inter-frame parallel, maximizes parallel degree, and is suitable for the design of software encoder and hardware encoder.

Description

Technical Field

[0001] This invention relates to digital video coding, and more particularly to a parallel method for the video coding process. Background Technology

[0002] In the field of digital imaging, still images are typically represented as a two-dimensional dot matrix, with each dot representing a pixel. Digital video or moving digital images are composed of a set of still digital images, each of which is called an image frame or video frame.

[0003] Please see Figure 1 The mainstream video or dynamic image compression algorithms currently include the following processes: prediction, transformation, quantization, in-loop filtering, entropy coding, and reconstruction, which are detailed below.

[0004] The prediction process primarily employs two methods: intra-frame prediction and inter-frame prediction. Intra-frame prediction utilizes the spatial correlation between adjacent pixels within the same video frame, using the values ​​of already encoded pixels to predict the values ​​of their neighboring pixels, thereby reducing the amount of information required. Inter-frame prediction utilizes the intra-frame correlation between adjacent video frames, using the pixel values ​​of already encoded video frames to predict the pixel values ​​of the current frame, also aiming to reduce the amount of information required.

[0005] The difference between the pixel value to be encoded and the predicted value is called the residual data, which is then fed into the transform process. The transform process converts the signal from the spatial domain to the frequency domain to remove spatial correlation. The spatial domain signal is transformed into a frequency domain signal, and the transformed signal is called the transform coefficients. The transform coefficients include low-frequency coefficients, mid-frequency coefficients, and high-frequency coefficients, with the energy primarily concentrated in the low-frequency coefficients.

[0006] Quantization is the process of dividing the transform coefficients by a certain integer, and it is divided into uniform quantization and non-uniform quantization. Coefficients with low energy in the transform coefficients (such as intermediate frequency coefficients and high frequency coefficients) can be removed by quantization without seriously affecting the visual quality of the reconstructed image. Therefore, the quantization process will produce quantization error.

[0007] Current mainstream video coding is block-based. Since errors occur during the encoding process, these errors can lead to distortion. To reduce the visual degradation caused by distortion, in-loop filtering is introduced in video coding to improve video quality. A filtering mode is selected during video encoding, and this selected filtering mode is executed during frame reconstruction or decoding.

[0008] Entropy coding further compresses and encodes the final coefficient and information data generated in the above process to remove statistical redundancy. Entropy coding is lossless coding, and its output is the final output of the video encoding process, containing all the information from the video decoding process.

[0009] Because of the precision loss during video encoding, in order to ensure that the encoding and decoding processes have the same baseline, the encoded frames need to be reconstructed during the encoding process. The reconstructed frames serve as a reference for subsequent inter-frame prediction and are called reference frames. The reconstruction process consists of inverse quantization, inverse transform, prediction compensation, and filtering. The first three are the inverse operations of the corresponding steps in the encoding process, and the reconstruction process is consistent with the actual decoding process.

[0010] exist Figure 1 In the video encoding process shown, each step (process) is performed sequentially and is interdependent. For example, intra-frame prediction requires image information from adjacent positions to complete the prediction of the current position, while inter-frame prediction requires reconstruction information from other frames to complete the prediction of the current frame; intra-loop filtering requires image information from adjacent positions to calculate the correction value; in the entropy coding process, in order to improve compression efficiency, it is necessary to establish a coding context through the encoded data, which also depends on the encoded images within the frame.

[0011] Video encoding is a computationally intensive process. Therefore, in addition to software video encoders running on general-purpose CPUs, the industry also employs various hardware acceleration devices (such as GPUs and FPGAs) to speed up the encoding process, or designs hardware encoders (dedicated ASIC chips) to achieve higher video encoding efficiency. With the increasing multi-core nature of various hardware devices, a method to better utilize hardware resources and execute multiple encoding tasks simultaneously to improve video encoding efficiency is called parallel video encoding. In parallel computing, independent tasks can execute concurrently, while dependent tasks must wait for the tasks they depend on to complete, which reduces the overall parallelism of the task. Therefore, maximizing or avoiding dependencies in the video encoding process is the first problem that parallel video encoding needs to solve. Since there are inevitably correlations between the various processes in video encoding, dependencies cannot be completely eliminated. How to coordinate and synchronize multiple tasks is the second problem that parallel video encoding needs to solve.

[0012] Please see Figure 2 Video encoding employs a block-based coding method, where each frame of the image to be encoded is divided into multiple non-overlapping coding units. Examples include H.264's MB (macro block, a 16-pixel x 16-pixel image block) and H.265's CTU (coding tree unit, supporting a maximum of 64-pixel x 64-pixel image blocks). When encoding a frame, each coding unit within the frame undergoes the aforementioned encoding process sequentially from left to right and from top to bottom.

[0013] When there are multiple independent processing units, a pipelined parallel coding method can be used, such as... Figure 3 As shown. Figure 3 Taking a system with four independent computational units as an example, four coding blocks can be encoded simultaneously. The encoding process of the second coding block is one step slower than that of the first, the third coding block is one step slower than that of the second, and so on. Because filtering and entropy coding are lossless—meaning that after performing the forward and reverse transformations, the result is the same as the original value—the reconstruction process omits this step. It is essentially starting from the quantized value, performing inverse quantization -> inverse transform -> reconstruction. Figure 1 The in-loop filtering and filtering shown are usually performed together in actual implementation; that is, during the filtering process, firstly, the filtering mode is determined and output to the entropy encoder, and secondly, the filtered pixels are determined for reconstruction. The main problem with this pipelined parallel encoding method is the low parallel efficiency when the pipeline is established or about to end, and since hardware encoders have usually adopted this method, how to further improve the parallel efficiency of the encoder remains a problem to be solved.

[0014] As mentioned earlier, video encoding requires strict adherence to the order of encoded blocks. Due to dependencies between steps (processes), complete parallelism cannot be achieved between encoded blocks within the same frame. To improve parallelism, relevant video standards have incorporated independently coded and decodeable units, such as SLICE in H.264 and TILE and SLICE defined in H.265. SLICE and TILE refer to units that can be used to divide a frame of image into multiple independently coded and decodeable units according to certain rules. A TILE can be further divided into one or more SLICE, and these units are independent of each other. Please refer to [link to relevant documentation]. Figure 4 Each small square represents a coded block. Long dashed lines are the boundaries of a TILE; an image frame is divided into two TILEs. Thick solid lines and short dashed lines are the boundaries of a SLICE; a TILE is divided into multiple SLICEs. Figure 4 The gray shading in the image schematically represents three SLICEs. The division of SLICEs and TILEs sacrifices some coding efficiency to achieve parallelism. However, the H.265 standard only defines the basic methods and constraints for SLICE and TILE division, without specifying the concrete implementation and parallel processing methods. Summary of the Invention

[0015] The technical problem to be solved by this invention is to propose a method for implementing intra-frame parallel coding and inter-frame parallel coding of video.

[0016] To address the aforementioned technical problems, this invention discloses a method for implementing parallel video coding, comprising the following steps: Step S1: Divide the video frame using SLICE as defined in the H.264 standard; the resulting SLICE is called a subframe. Alternatively, divide the video frame using TILE and / or SLICE as defined in the H.265 standard; the smallest of the resulting TILE or SLICE is called a subframe. Step S2: During intra-frame parallel coding, each subframe is encoded independently; when all subframes are encoded, the current frame is encoded. When performing inter-frame parallel coding, the integer pixel motion estimation of the current frame's coded block (m,n) is performed after the coded block rows containing the coded blocks (m,m+k) of all reference frames have been encoded. The sub-pixel motion estimation of the current frame's coded block (m,n) is performed after the coded block rows containing the coded blocks (m,m+k+N) of all reference frames have been encoded. The coded block (m,n) refers to the x-axis offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame, with an offset of m and a y-axis offset of n. When using H.264 encoding, N = 16; when using H.265 encoding, N = 8, 16, 32, or 64.

[0017] Furthermore, the subframe satisfies the following conditions: (1) The subframe consists of complete coded blocks; the SLICE either contains a complete row of coded blocks in a video frame or contains a complete row of coded blocks in a TILE; (2) The SLICE and TILE are divided as evenly as possible, that is, each SLICE or TILE contains the same number of coded blocks as much as possible.

[0018] Further, the video frame sub-frame division in step S1 is divided into the following three cases: Case 1: The video frame is divided using the SLICE defined in the H.264 standard, and each SLICE is called a subframe; or the video frame is divided using the SLICE defined in the H.265 standard, and each SLICE is called a subframe. Case 2: The video frame is divided using the TILE defined in the H.265 standard, and each TILE is called a subframe. Case 3: The video frame is divided using both TILE and SLICE defined in the H.265 standard, and each TILE is further divided into multiple SLICEs, and each SLICE is called a subframe.

[0019] Furthermore, in step S2, intra-frame parallel coding and inter-frame parallel coding are independent of each other and are divided into the following working modes: (1) intra-frame parallel coding only; (2) inter-frame parallel coding only; (3) both intra-frame parallel coding and inter-frame parallel coding are present.

[0020] Furthermore, in step S2, when performing inter-frame parallel coding, the motion search range of the coding block row where the current frame's coding block (m,n) is located on the reference frame is from the coding block (m,nk) of the reference frame to the coding block row where the coding block (m,n+k) is located; the minimum value of k is 8, and the maximum value is 1 / 4 of the video frame height.

[0021] Preferably, in step S2, when performing inter-frame parallel coding, the integer pixel motion estimation of the LCU(m,n) of the current frame is performed after the LCU rows containing the LCU(m,m+k) of all reference frames have been encoded, and the sub-pixel motion estimation of the LCU(m,n) of the current frame is performed after the LCU rows containing the LCU(m,m+k+N) of all reference frames have been encoded; LCU(m,n) refers to the x-axis offset of the top-left pixel of the LCU relative to the top-left pixel of the video frame, with an offset of m and a y-axis offset of n; when using H.264 encoding, the size of the LCU is a 16-pixel × 16-pixel coding block; when using H.265 encoding, the size of the LCU is a 64-pixel × 64-pixel coding block.

[0022] Furthermore, in step S2, a two-dimensional array is set for each reference frame. The number and distribution of elements in the two-dimensional array correspond one-to-one with the number and distribution of each coded block line in each subframe of the reference frame. Each element corresponds to a coded block line in a subframe. When a coded block line in a subframe of the reference frame is completed, the value of the corresponding element in the two-dimensional array is incremented by 1. When the current video frame is being encoded, the value of each element in the two-dimensional array corresponding to the reference frame is checked to determine the encoding progress of each coded block line in the subframe of the reference frame, thereby determining whether the encoding conditions of the current video frame are met.

[0023] Further, in step S2, when encoding a coding block line begins, the motion search range [ymin, ymax] of the current coding block line in the reference frame is first determined; any coding block in the current coding block line is represented as coding block (m, n), then ymin = nk, ymax = n + k; for all reference frames, if the motion search range is located inside a subframe of the reference frame, then the maximum y-axis coordinate corresponding to the encoded coding block line in that subframe is ≥ ymax and is considered to meet the conditions for motion search; if the motion search range spans multiple subframes of the reference frame, then the subframe to which ymin belongs (inclusive) to the subframe to which ymax belongs (exclusive) has been fully encoded, and the maximum y-axis coordinate corresponding to the encoded coding block line in the subframe to which ymax belongs is ≥ ymax, and is considered to meet the conditions for motion search; if the conditions for motion search are not met, then wait until the conditions are met before starting the encoding of the current coding block line.

[0024] Furthermore, in step S2, when there are multiple subframes in the horizontal direction of the reference frame, the slowest subframe is used as the comparison benchmark.

[0025] This invention also discloses a device for implementing parallel video coding, including a subframe partitioning unit and a parallel processing unit. The subframe partitioning unit is used to partition video frames using SLICE as defined in the H.264 standard, and the resulting SLICE is called a subframe; or to partition video frames using TILE and / or SLICE as defined in the H.265 standard, and the smallest of the resulting TILE or SLICE is called a subframe. The parallel processing unit is used to encode each subframe independently when performing intra-frame parallel coding; the current frame is encoded when all subframes are encoded; the parallel processing unit 2 is also used to perform integer pixel motion estimation of the current frame's coded block (m,n) after all reference frames' coded blocks (m,m+k) have been encoded when performing inter-frame parallel coding, and to perform sub-pixel motion estimation of the current frame's coded block (m,n) after all reference frames' coded blocks (m,m+k+N) have been encoded; the coded block (m,n) refers to the x-axis offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame, with a y-axis offset of n; when using H.264 encoding, N=16; when using H.265 encoding, N=8, 16, 32, or 64.

[0026] This invention proposes a scheme that simultaneously supports intra-frame parallelism and inter-frame parallelism in video coding, including subframe partitioning methods, inter-frame parallel dependency removal methods, and synchronization mechanisms between parallel tasks. This method can further improve parallel coding efficiency based on the aforementioned pipelined parallel methods. This scheme is applicable to the design of parallel tasks for both software encoders and hardware encoders. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall process of existing video encoding.

[0028] Figure 2 This is a schematic diagram illustrating the division of image frames into image blocks during the video encoding process.

[0029] Figure 3 This is a schematic diagram of an existing video encoding method that uses a pipelined parallel approach.

[0030] Figure 4 This is a schematic diagram of TILE and SLICE as defined in the H.265 standard.

[0031] Figure 5 This is a flowchart illustrating the implementation method of video parallel coding proposed in this invention.

[0032] Figures 6 to 8 This is a schematic diagram illustrating three scenarios in which the present invention divides a video frame into subframes.

[0033] Figure 9 This is a schematic diagram of inter-frame reference.

[0034] Figure 10 This is a schematic diagram of the structure of the video parallel coding implementation device proposed in this invention.

[0035] The accompanying figures are labeled as follows: 1 represents a subframe division unit, and 2 represents a parallel processing unit. Detailed Implementation

[0036] Please see Figure 5 The video parallel coding implementation method proposed in this invention includes the following steps.

[0037] Step S1: Divide the video frame using SLICE as defined in the H.264 standard. The resulting SLICE is called a subframe. Alternatively, divide the video frame using TILE and / or SLICE as defined in the H.265 standard. The smallest TILE or SLICE is called a subframe.

[0038] The sub-frames must meet the following conditions: (1) A sub-frame must consist of complete coded blocks and cannot contain incomplete coded blocks; a sub-frame obtained by using the SLICE mechanism must either contain a complete row of coded blocks from a video frame or a complete row of coded blocks from a TILE. (2) Under the premise of meeting the first condition, SLICE and TILE should be divided as evenly as possible, that is, each SLICE or TILE should contain the same number of coded blocks as much as possible. The more evenly they are divided, the better the parallelism. For example, if a video frame contains 7×4 coded blocks, when it is divided into two sub-frames, it cannot be divided evenly. It can only be divided into two sub-frames consisting of 3×4 coded blocks and 4×4 coded blocks, which are as equal as possible. The advantage of this division is that each sub-frame contains as many coded blocks as possible, so that the amount of computation for encoding is as consistent as possible, thereby reducing the waiting time of the task.

[0039] This step of dividing the video frame into subframes can be divided into the following three cases.

[0040] Scenario 1, such as Figure 6 As shown, each small square represents a coding block, and the short dashed lines are the boundaries of the SLICE. Video frames are divided using the SLICE defined in the H.264 standard, and each resulting SLICE is called a subframe; alternatively, video frames are divided using the SLICE defined in the H.265 standard, and each resulting SLICE is called a subframe.

[0041] Scenario 2, such as Figure 7 As shown, each small square represents a coding block, and the long dashed line is the boundary of the TILE. Video frames are divided using TILEs as defined in the H.265 standard, and each resulting TILE is called a subframe.

[0042] Scenario 3, such as Figure 8 As shown, each small square represents a coding block, the long dashed line is the boundary of the TILE, and the short dashed line is the boundary of the SLICE. Video frames are divided using TILEs and SLICEs as defined in the H.265 standard. Each TILE is further divided into multiple SLICEs, and each resulting SLICE is called a subframe.

[0043] The concepts of TILE and SLICE defined in the technical specifications are quite broad, supporting very flexible partitioning methods. The three partitioning methods supported by this invention are a subset of all partitioning methods in the technical specifications, and are also the most suitable partitioning methods selected by this invention in the prior art.

[0044] Step S2: During intra-frame parallel coding, each subframe is encoded independently. Once all subframes have been encoded, the current frame is complete.

[0045] When performing inter-frame parallel coding, the integer pixel motion estimation of the coded block (m,n) of the video frame to be encoded is performed after the coded block rows containing the coded blocks (m,m+k) of all reference frames have been encoded. The sub-pixel motion estimation of the coded block (m,n) of the video frame to be encoded is performed after the coded block rows containing the coded blocks (m,m+k+N) of all reference frames have been encoded. The coded block (m,n) refers to the top-left pixel of this coded block, whose x-axis (video frame width) offset is m and whose y-axis (video frame height) offset is n relative to the top-left pixel of the video frame.

[0046] Intra-frame parallelism and inter-frame parallelism are independent of each other and can have the following working modes: (1) only intra-frame parallelism; (2) only inter-frame parallelism; (3) both intra-frame parallelism and inter-frame parallelism.

[0047] Please see Figure 9When encoding a coded block in the current frame, the pixel values ​​of the current coded block may be predicted using the pixel values ​​of one or more coded blocks in reference frames (called reference blocks). The process of searching for a reference block in a reference frame is called motion estimation (ME), and the search range is called the motion search range. ME is a crucial step in video coding; the higher the similarity between the found reference block and the current coded block, the fewer residuals need to be encoded, and the higher the coding quality. Therefore, the main goal of motion estimation is to find the best reference block in the best reference frame from the list of reference frames for the current frame.

[0048] If the current frame and the reference frame are encoded in parallel, it is necessary to ensure that all coded blocks within the search range of the reference frame are encoded before the current coded block begins encoding. It is easy to see that, all other things being equal, the smaller the motion search range, the higher the inter-frame parallelism achievable, but the probability of finding the best matching block also decreases, affecting the encoding quality; conversely, the lower the inter-frame parallelism achievable, the higher the encoding quality. In practical applications, motion estimation consumes a large amount of computation, and computing resources are always limited, so the motion search range is usually restricted to a certain range. This invention, considering all aspects, restricts the motion search range to a range of k pixels up and down, with no horizontal restriction. The minimum value of k is 8, and the maximum value is 1 / 4 of the video frame height (in pixels). Within this range, inter-frame parallelism is good. For example, a value of 64 for k.

[0049] The coded block (m,n) represents the offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame. Within the aforementioned motion search range, the search range for the row of coded blocks containing coded block (m,n) is from the row of coded blocks containing coded block (m,nk) of the reference frame to the row of coded blocks containing coded block (m,n+k). Therefore, coded block (m,n) must wait for the row of coded blocks containing coded block (m,n+k) of the reference frame to complete encoding before it can begin encoding. Because the row direction is unrestricted, other coded blocks in the row of coded block (m,n) do not need to be synchronized again (synchronization means waiting, or relying on other tasks to complete). Therefore, the prerequisite for the row of coded blocks containing coded block (m,n) to begin encoding (integer pixel motion estimation) is that the row of coded blocks containing coded block (m,n+k) in the reference frame has completed encoding.

[0050] The above is integer pixel motion estimation, without considering subpixel motion estimation. Because subpixels are not actual pixels, they need to be obtained through interpolation using adjacent actual pixel values. Therefore, in the case of subpixel motion estimation, the motion search of the coding block line containing coded block (m,n) requires some pixels from the coding block line containing coded block (m,n+k+N). Thus, the prerequisite for starting encoding (subpixel motion estimation) in the coding block line containing coded block (m,n+k+N) on the reference frame is that the coding block line containing coded block (m,n+k+N) has completed encoding. Here, N is the height of the coding block (in pixels), N = 16 on H.264, and N = 8, 16, 32, or 64 on H.265.

[0051] Therefore, the motion search range limitation method in this invention can not only realize inter-frame parallel coding, but also only requires synchronization once before the start of the coding block at the beginning of the coding block line, thus reducing the frequency of parallel task synchronization.

[0052] Preferably, in step S2, when performing inter-frame parallel coding, the integer pixel motion estimation of the LCU(m,n) of the video frame to be encoded is performed after the LCU rows containing the LCUs(m,m+k) of all reference frames have been encoded, and the sub-pixel motion estimation of the LCU(m,n) of the video frame to be encoded is performed after the LCU rows containing the LCUs(m,m+k+N) of all reference frames have been encoded. LCU(m,n) refers to the offset of the top-left pixel of the LCU relative to the top-left pixel of the video frame along the x-axis (video frame width direction) and the y-axis (video frame height direction). When using H.264 encoding, the LCU size is a 16-pixel × 16-pixel coding block. When using H.265 encoding, the LCU size is a 64-pixel × 64-pixel coding block.

[0053] In step S2, to achieve frame-level parallelism (i.e., frame parallelism) synchronization, a two-dimensional array is set for each reference frame. The number and distribution of elements in this two-dimensional array correspond one-to-one with the number and distribution of each coded block line in each subframe of the reference frame, with each element corresponding to a coded block line in a subframe. When a coded block line in a subframe of the reference frame is completed, the value of the corresponding element in the two-dimensional array is incremented by 1. When the current video frame is being encoded, checking the values ​​of each element in the two-dimensional array corresponding to the reference frame reveals the encoding progress of each coded block line in the subframe of the reference frame, thereby determining whether the encoding conditions of the current video frame are met.

[0054] In step S2, when encoding a coding block line begins, the motion search range [ymin, ymax] of the current coding block line in the reference frame is first determined. If any coding block in the current coding block line is represented as coding block (m, n), then ymin = nk, ymax = n + k. The motion search ranges for integer pixel motion estimation and sub-pixel motion estimation are the same, the difference being: integer pixels are existing pixels in the reference frame; sub-pixels are non-existent pixels, calculated based on the integer pixel values ​​of their surroundings using an interpolation algorithm. Currently, commonly used interpolation algorithms include 6-tap, 7-tap, or 8-tap interpolation filtering algorithms. Therefore, sub-pixels near the boundary require integer pixel values ​​outside the boundary, thus expanding the waiting range. For all reference frames, if the motion search range is located within a sub-frame of the reference frame, then it must be ensured that the maximum y-axis coordinate corresponding to the encoded coding block line in that sub-frame is ≥ ymax. Since the y-axis coordinates of video frames increase from top to top, the conditions for motion search are met. If the motion search range spans (covers) multiple subframes on the reference frame, then it must be ensured that the subframe from which ymin belongs (inclusive) to the subframe from which ymax belongs (exclusive, i.e., not all subframes of ymax need to be encoded) has been fully encoded, and the maximum y-axis coordinate corresponding to the encoded block line in the subframe of ymax must be greater than or equal to ymax. Only then are the conditions for motion search met. If the conditions for motion search are not met, it is necessary to wait until the conditions are met before encoding of the current block line can begin.

[0055] When there are multiple subframes in the horizontal direction of the reference frame, the slowest subframe should be used as the comparison benchmark. All subframes in the same horizontal direction may start encoding simultaneously or not. Even if they start simultaneously, their progress cannot be guaranteed to be completely consistent. Therefore, only when the slowest subframe meets the condition can it be considered that the condition has truly been met. For example, if the reference frame is divided into left and right subframes, the waiting strategy of this invention is that the entire coding block line containing ymax must be encoded. This requires the coding block lines of both the left and right subframes containing ymax to be fully completed.

[0056] Please see Figure 10 The video parallel coding implementation device proposed in this invention includes a subframe partitioning unit 1 and a parallel processing unit 2. Figure 10 The device shown is Figure 5 The method shown corresponds to this.

[0057] The subframe division unit 1 is used to divide video frames using SLICE (strips) defined in the H.264 standard, and the resulting SLICE is called a subframe; or to divide video frames using TILE (pieces) and / or SLICE defined in the H.265 standard, and the smallest of the resulting TILE or SLICE is called a subframe.

[0058] The sub-frame must satisfy the following conditions: (1) The sub-frame must consist of complete coded blocks and cannot contain incomplete coded blocks; the sub-frame obtained by using the SLICE mechanism must either contain a complete line of coded blocks in a video frame or a complete line of coded blocks in a TILE. (2) Under the premise of satisfying the first condition, the SLICE and TILE should be divided as evenly as possible, that is, each SLICE or TILE should contain the same number of coded blocks as much as possible.

[0059] The parallel processing unit 2 is used to encode each subframe independently during intra-frame parallel coding. The current frame is completed when all subframes are encoded. The parallel processing unit 2 is also used to perform integer pixel motion estimation of the coded block (m,n) of the video frame to be encoded after the coded block rows (m,m+k) of all reference frames have been encoded, and to perform sub-pixel motion estimation of the coded block (m,n) of the video frame to be encoded after the coded block rows (m,m+k+N) of all reference frames have been encoded. The coded block (m,n) refers to the x-axis (video frame width direction) offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame, with an offset of m and n along the y-axis (video frame height direction). When using the H.264 standard for video encoding, N = 16; when using the H.265 standard for video encoding, N = 8, 16, 32, or 64.

[0060] Compared with existing technologies, this invention achieves intra-frame parallel coding and inter-frame parallel coding, maximizing parallelism and making it suitable for both software and hardware encoder design. This invention is applicable to current mainstream video coding formats, has a wide range of applications and strong scalability, while avoiding the losses and reduced parallelism caused by frequent synchronization of parallel tasks.

[0061] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for implementing parallel video coding, characterized in that, Includes the following steps; Step S1: Divide the video frame using the SLICE defined in the H.264 standard. The resulting SLICE is called a subframe; or divide the video frame using the TILE and / or SLICE defined in the H.265 standard. The smallest TILE or SLICE is called a subframe. The subframe satisfies the following conditions: (1) The subframe consists of complete coded blocks; the SLICE either contains a complete row of coded blocks in a video frame or contains a complete row of coded blocks in a TILE; (2) The SLICE and TILE are divided as evenly as possible, that is, each SLICE or TILE contains the same number of coded blocks as much as possible. Step S2: When performing intra-frame parallel coding, each subframe is encoded independently; when all subframes have been encoded, the current frame is encoded. When performing inter-frame parallel coding, the integer pixel motion estimation of the current frame's coded block (m, n) is performed after the coded block rows containing the coded blocks (m, m+k) of all reference frames have been encoded. The sub-pixel motion estimation of the current frame's coded block (m, n) is performed after the coded block rows containing the coded blocks (m, m+k+N) of all reference frames have been encoded. The coded block (m, n) refers to the x-axis offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame, with an offset of m and a y-axis offset of n. The minimum value of k is 8, and the maximum value is 1 / 4 of the video frame height. When using H.264 encoding, N=16. When using H.265 encoding, N=8, 16, 32, or 64.

2. The method for implementing parallel video coding according to claim 1, characterized in that, In step S1, the video frame is divided into sub-frames in the following three cases; Scenario 1: The video frames are divided using the SLICE defined in the H.264 standard, and each SLICE is called a subframe; or the video frames are divided using the SLICE defined in the H.265 standard, and each SLICE is called a subframe. Scenario 2: The video frames are divided using the TILE defined in the H.265 standard, and each TILE is called a subframe. Scenario 3: The video frames are divided using TILE and SLICE as defined in the H.265 standard. Each TILE is further divided into multiple SLICE, and each SLICE is called a subframe.

3. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, intra-frame parallel coding and inter-frame parallel coding are independent of each other and are divided into the following working modes: (1) intra-frame parallel coding only; (2) inter-frame parallel coding only; (3) intra-frame parallel coding and inter-frame parallel coding are both present.

4. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, when performing inter-frame parallel coding, the motion search range of the coding block row where the current frame's coding block (m,n) is located on the reference frame is from the coding block (m,nk) of the reference frame to the coding block row where the coding block (m,n+k) is located.

5. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, when performing inter-frame parallel coding, the integer pixel motion estimation of the LCU(m,n) of the current frame is performed after the LCU rows containing the LCU(m,m+k) of all reference frames have been encoded, and the sub-pixel motion estimation of the LCU(m,n) of the current frame is performed after the LCU rows containing the LCU(m,m+k+N) of all reference frames have been encoded. LCU(m,n) refers to the x-axis offset of the top-left pixel of the LCU relative to the top-left pixel of the video frame, with an offset of m and a y-axis offset of n. When H.264 encoding is used, the size of the LCU is a 16-pixel × 16-pixel coding block. When H.265 encoding is used, the size of the LCU is a 64-pixel × 64-pixel coding block.

6. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, a two-dimensional array is set for each reference frame. The number and distribution of elements in the two-dimensional array correspond one-to-one with the number and distribution of each coded block line in each subframe of the reference frame. Each element corresponds to a coded block line in a subframe. When a coded block line in a subframe of the reference frame is completed, the value of the corresponding element in the two-dimensional array is incremented by 1. When the current video frame is being encoded, the value of each element in the two-dimensional array corresponding to the reference frame is checked to determine the encoding progress of each coded block line in the subframe of the reference frame, thereby determining whether the encoding conditions of the current video frame are met.

7. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, when encoding a coding block line begins, the motion search range [ymin, ymax] of the current coding block line in the reference frame is first determined. Any coding block in the current coding block line is represented as coding block (m, n), then ymin = nk, ymax = n + k. For all reference frames, if the motion search range is located inside a subframe of the reference frame, then the maximum y-axis coordinate corresponding to the encoded coding block line in that subframe is ≥ ymax, which is considered to meet the conditions for motion search. If the motion search range spans multiple subframes of the reference frame, then the subframe to which ymin belongs (inclusive) to which ymax belongs (exclusive) has been fully encoded, and the maximum y-axis coordinate corresponding to the encoded coding block line in the subframe to which ymax belongs is ≥ ymax, which is considered to meet the conditions for motion search. If the conditions for motion search are not met, then wait until the conditions are met before starting the encoding of the current coding block line.

8. The method for implementing parallel video coding according to claim 1, characterized in that, In step S2, when there are multiple subframes in the horizontal direction of the reference frame, the slowest subframe is used as the comparison benchmark.

9. A device for implementing parallel video coding, characterized in that, Includes subframe partitioning units and parallel processing units; The subframe partitioning unit is used to partition video frames using SLICE as defined in the H.264 standard, and the resulting SLICE is called a subframe; or to partition video frames using TILE and / or SLICE as defined in the H.265 standard, and the smallest of the resulting TILE or SLICE is called a subframe. The subframe satisfies the following conditions: (1) The subframe consists of complete coded blocks; the SLICE either contains a complete row of coded blocks in a video frame or contains a complete row of coded blocks in a TILE; (2) The SLICE and TILE are divided as evenly as possible, that is, each SLICE or TILE contains the same number of coded blocks as much as possible. The parallel processing unit is used to encode each subframe independently during intra-frame parallel coding; the current frame is encoded when all subframes are encoded. The parallel processing unit is also used to perform integer pixel motion estimation of the current frame's coded block (m, n) during inter-frame parallel coding after the coded block rows (m, m+k) of all reference frames have been encoded, and to perform sub-pixel motion estimation of the current frame's coded block (m, n) after the coded block rows (m, m+k+N) of all reference frames have been encoded. The coded block (m, n) refers to the x-axis offset of the top-left pixel of the coded block relative to the top-left pixel of the video frame, with a y-axis offset of n. The minimum value of k is 8, and the maximum value is 1 / 4 of the video frame height. When using H.264 encoding, N=16; when using H.265 encoding, N=8, 16, 32, or 64.

Images