Video encoding apparatus with limited reconstruction buffer and associated video encoding method

Abstract

A video encoding apparatus including a reconstruction buffer with fixed size and / or bandwidth limitations, and an associated video encoding method. Specifically, a video encoding apparatus includes a data buffer and video encoding circuitry. Encoding of a first frame includes: deriving reference pixels of a reference frame from the reconstructed pixels of the first frame, and storing the reference pixel data in the data buffer for inter-frame prediction, wherein the reference pixel data includes information about the pixel values ​​of the reference pixels. Encoding of a second frame includes: performing prediction on coding units in the second frame to determine a target prediction factor for the coding unit. The prediction step performed on the coding unit includes: determining the target prediction factor of the coding unit based on whether a search range for the prediction factor of the coding unit for the reference frame includes at least one reference pixel of the reference frame that cannot be accessed by the video encoding circuitry.

Description

[0001] Related applications

[0002] This application is a divisional application of patent application number 202010217722.8, entitled "Video Coding Device with Reconstruction Buffer and Associated Video Coding Method". Technical Field

[0003] This invention relates to video coding, and more specifically, to a video coding apparatus including a reconstruction buffer with a fixed size and / or bandwidth limitation, and an associated video coding method. Background Technology

[0004] Conventional video coding standards typically employ coding unit (CU)-based techniques to leverage spatial and temporal redundancy. For example, this basic approach divides the entire source frame into multiple CUs, performs intra-frame / inter-frame prediction on each CU, transforms the residuals of each CU, and performs quantization and entropy coding. Furthermore, a reconstructed frame is generated to serve as a reference frame, providing reference pixel data used to encode subsequent CUs. For some video coding standards, in-loop filters can be used to enhance the image quality of the reconstructed frame / reference frame. A video decoder is used to perform the inverse operations of the video coding performed by the video encoder. For example, the video decoder generates a reconstructed frame to serve as a reference frame, providing reference pixel data used to decode subsequent CUs.

[0005] In conventional video coding systems, a reconstruction / reference buffer is used to buffer the reconstructed frames / reference frames that can be used for inter-frame prediction. Typically, the reconstruction / reference buffer is implemented by allocating a buffer in dynamic random access memory (DRAM). This allows the size of the reconstruction / reference buffer to vary depending on the resolution of the input source frames. A variable-size buffer can fully store the information of the reconstructed frames / reference frames. As a result, conventional video coding system designs require high DRAM bandwidth and have high power consumption. Summary of the Invention

[0006] One of the objectives of the claimed invention is to provide a video encoding apparatus including a reconstruction buffer with fixed size and / or bandwidth limitations, and an associated video encoding method.

[0007] According to a first aspect of the present invention, an exemplary video encoding apparatus is disclosed. The exemplary video encoding apparatus includes a data buffer and a video encoding circuit. The video encoding circuit is configured to encode a plurality of frames into a bitstream, wherein each frame includes a plurality of encoding units, each encoding unit includes a plurality of pixels, and the frames include a first frame and a second frame. The encoding operation of the first frame includes: deriving a plurality of reference pixels of a reference frame from a plurality of reconstructed pixels of the first frame; and storing the reference pixel data in the data buffer for inter-frame prediction, wherein the reference pixel data includes information about the pixel values ​​of the reference pixels. The encoding operation of the second frame includes: performing a prediction on the encoding units in the second frame to determine a target predictor for the encoding units. The prediction step performed on the encoding units includes: determining the target predictor of the encoding units based on whether a search range for the predictor of the encoding unit for the reference frame includes at least one reference pixel of the reference frame that cannot be accessed by the video encoding circuit.

[0008] According to a second aspect of the present invention, an exemplary video coding method for encoding multiple frames into a bitstream is disclosed, wherein each frame includes multiple coding units, and each coding unit includes multiple pixels. The video coding method includes the steps of: encoding a first frame included in the frame, and encoding a second frame included in the frame. The step of encoding the first frame includes: deriving multiple reference pixels of a reference frame from multiple reconstructed pixels of the first frame; and storing the reference pixel data in a data buffer for inter-frame prediction, wherein the reference pixel data includes information about the pixel values ​​of the reference pixels. The step of encoding the second frame includes: performing prediction on the coding units in the second frame to determine a target prediction factor for the coding unit, wherein the prediction step of performing prediction on the coding unit includes: determining the target prediction factor of the coding unit based on whether a search range for the prediction factor of the coding unit for the reference frame includes at least one reference pixel of the reference frame that cannot be accessed by the video coding circuit.

[0009] These and other objects of the invention will undoubtedly become apparent to those skilled in the art after reading the following detailed description of the preferred embodiments illustrated in the various figures. Attached Figure Description

[0010] Figure 1 This is a diagram illustrating a first video encoding device according to an embodiment of the present invention.

[0011] Figure 2 Example diagram of the prediction operation performed by prediction circuit 119 to find the predictor of the coding unit.

[0012] Figure 3 This is a diagram illustrating a second video encoding device according to an embodiment of the present invention.

[0013] Figure 4 This is a diagram illustrating a first example of performing prediction on a coding unit to determine a target predictor factor for the coding unit according to an embodiment of the present invention.

[0014] Figure 5 This is an example in Figure 4 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0015] Figure 6 This is a diagram illustrating a second example of performing prediction on a coding unit to determine a target predictor factor for the coding unit according to an embodiment of the present invention.

[0016] Figure 7 This is an example in Figure 6 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0017] Figure 8 This is a diagram illustrating a third example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention.

[0018] Figure 9 This is an example in Figure 8 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0019] Figure 10 This is a diagram illustrating a third video encoding device according to an embodiment of the present invention.

[0020] Figure 11 This is a diagram illustrating a first example of accessing a data buffer by compression / decompression according to an embodiment of the present invention.

[0021] Figure 12 This is a diagram illustrating a fourth video encoding device according to an embodiment of the present invention.

[0022] Figure 13 This is a diagram illustrating a second example of accessing a data buffer via compression / decompression according to an embodiment of the present invention.

[0023] Figure 14 This is a fourth example illustrating the performance of prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention.

[0024] Figure 15 This is an example in Figure 14 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0025] Figure 16 This is a diagram illustrating a third example of accessing a data buffer via compression / decompression according to an embodiment of the present invention.

[0026] Figure 17 This is a diagram illustrating a fourth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention.

[0027] Figure 18 This is a diagram illustrating a fifth example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention.

[0028] Figure 19 This is an example in Figure 18 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0029] Figure 20 This is a diagram illustrating a fifth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention.

[0030] Figure 21 This is a diagram illustrating a sixth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention.

[0031] Figure 22 This is a diagram illustrating a sixth example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention.

[0032] Figure 23 This is an example in Figure 22 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame.

[0033] Figure 24 This is a diagram illustrating a fifth video encoding device according to an embodiment of the present invention.

[0034] Figure 25 This is a diagram illustrating a seventh example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention.

[0035] Figure 26 This is an example in Figure 25 The diagram shows an example of motion vector search for a coding unit with a search range on a reference frame. Detailed Implementation

[0036] Throughout the following description and claims, certain terms are used to refer to specific components. As will be apparent to those skilled in the art, electronic device manufacturers may use different names to refer to components. This document is not intended to distinguish between components with different names rather than different functions. In the following description and claims, the term "comprising" is used in an open-ended manner and should therefore be interpreted as meaning "including, but not limited to...". Furthermore, the term "couple" is intended to refer to an indirect or direct electrical connection. Thus, if one device is coupled to another device, the connection can be a direct electrical connection or an indirect electrical connection via other devices and connectors.

[0037] Figure 1 This is a diagram illustrating a first video encoding apparatus according to an embodiment of the present invention. The video encoding apparatus 100 includes a video encoding circuit 102 and a data buffer 104. Regarding the video encoding circuit 102, the video encoding circuit includes: an adder 111 (the adder is used to perform data subtraction), a transform circuit (denoted by "T") 112, a quantization circuit (denoted by "Q") 113, an entropy encoding circuit (e.g., a variable length encoder) 114, an inverse quantization circuit (denoted by "IQ") 115, an inverse transform circuit (denoted by "IT") 116, an adder (the adder is used to perform data summation) 117, at least one in-loop filter (e.g., a deblocking filter) 118, a prediction circuit 119, and an intra / inter-frame mode selection switch 124. Prediction circuit 119 supports inter-frame prediction mode and intra-frame prediction mode, and includes inter-frame prediction circuit 120 (which includes motion estimation circuit (denoted by "ME") 121 and motion compensation circuit (denoted by "MC") 122) and intra-frame prediction circuit (denoted by "IP") 123. It should be noted that... Figure 1 The structure of the video encoding circuit 102 shown is merely illustrative and is not intended to limit the invention.

[0038] Adder 111 is a residual calculation circuit used to subtract the predicted block from the coding unit of the input source frame IMG to generate the residual of the coding unit to transform circuit 112. When the intra-frame / inter-frame mode selection switch 124 is controlled according to the intra-frame prediction mode, which is the optimal coding mode selected for the coding unit, the predicted block can be generated from the intra-frame prediction circuit 123, and when the intra-frame / inter-frame mode selection switch 124 is controlled according to the inter-frame prediction mode, which is the optimal coding mode selected for the coding unit, the predicted block can be generated from the inter-frame prediction circuit 120. After sequential processing by transform circuit 112 and quantization circuit 113, the residual of the coding unit is converted into quantized transform coefficients, which are then entropy-encoded into part of the bitstream BS at entropy coding circuit 114.

[0039] The video encoding circuit 102 has an internal decoding circuit. Therefore, the quantized transform coefficients are sequentially processed via inverse quantization circuit 115 and inverse transform circuit 116 to generate the decoded residual of the coding unit in adder 117. Adder 117 acts as a reconstruction circuit that combines the decoded residual of the coding unit with the predicted block of the coding unit to generate the reconstructed coding unit in the reconstructed frame. Before storing the reconstructed frame in data buffer 104, an in-loop filter 118 can perform specified in-loop filtering on the reconstructed frame. For example, in-loop filter 118 may include a deblocking filter designed to reduce blocking artifacts introduced due to CU-based coding.

[0040] In this example, reference pixels for a reference frame are derived from the reconstructed pixels of the input source frame IMG, and the reference pixel data is stored in a data buffer 104, which acts as a reconstruction / reference buffer. The reference pixel data includes information about the pixel values ​​of the reference pixels. The video coding circuit 102 (specifically, the inter-frame prediction circuit 120) can retrieve a portion of the reference pixel data stored in the data buffer 104 to find the prediction factor for the coding unit in inter-frame prediction mode.

[0041] Figure 2 This is a diagram illustrating the prediction operation performed by prediction circuit 119 to find predictor factors for coding units. The video coding circuit 102 is configured to encode multiple input source frames IMG into a bitstream BS, wherein each frame includes multiple coding units, and each coding unit includes multiple pixels. Figure 2As shown, the input source frame IMG includes a first frame IMG1 and a second frame IMG2. Taking the first frame IMG1 as an example, the first frame is divided into multiple coding units 201, each coding unit having multiple pixels 202. The first frame IMG1 and the second frame IMG2 are consecutive frames fed into the video coding circuit 102. Therefore, the start time for encoding the first frame IMG1 is earlier than the start time for encoding the second frame IMG2. The video coding circuit 102 encodes the first frame IMG1 into a bitstream BS and further generates a reconstructed frame at the reconstruction circuit (i.e., adder 117). The reference frame IMG_R can be derived by passing the reconstructed frame (which is generated based on the encoded data of the first frame IMG1) through the in-loop filter 118. The current coding unit 204 of the second frame IMG2 is encoded using one of the prediction methods supported by the prediction circuit 119. When the optimal coding mode selected for the current coding unit 204 is intra-frame prediction mode, Palette mode, or intra-block copy mode, the prediction factor of the current coding unit 204 can be derived from previously reconstructed pixels 208 in the same frame (i.e., the second frame IMG2). In another case, where the optimal coding mode selected for the current coding unit 204 is inter-frame prediction mode, the prediction factor of the current coding unit 204 can be derived from reference pixels in other frames (e.g., the reference frame IMG_R). The search range SR of the current coding unit 204... W,H It is limited by two parameters, W and H. More specifically, the search range SR W,H It is limited by (±W, ±H). The co-located block 206 can be found in the reference frame IMG_R. The search range SR assigned to the current coding unit 204 is... W,H It is applied to the reference frame IMG_R, centered on the co-located block 206. Therefore, the search range SR in the reference frame IMG_R is... W,H This defines the boundaries of the motion vectors and limits the number of reference blocks to be evaluated. In other words, only the frames IMG_R within the search range SR are referenced. W,H Reference pixels within the unit can be referenced by motion estimation to find the predictor (i.e., motion vector) of the current coding unit 204.

[0042] To reduce DRAM bandwidth and / or power consumption, a static random access memory (SRAM) can be used to implement a reconstruction / reference buffer (e.g., data buffer 104). However, a memory bandwidth (BW) limit can still be applied to the data buffer 104. Based on the memory BW limit, reading from the data buffer 104 and sending one or more reference pixels stored in the data buffer 104 to the video encoding circuit 102 (specifically, the motion estimation circuit 121 of the prediction circuit 119) can be prevented. In other words, the reference pixels stored in the data buffer 104 include: valid reference pixels accessible to the video encoding circuit 102, and invalid reference pixels inaccessible to the video encoding circuit 102. In this embodiment, the video encoding circuit 102 (specifically, the prediction circuit 119)... Figure 2 The current coding unit 204 in the second frame IMG2 shown performs prediction to determine the target prediction factor for the current coding unit 204. Since one or more reference pixels stored in the data buffer 104 may be invalid reference pixels, the video coding circuit 102 (specifically, the prediction circuit 119) checks the search range SR against the reference frame IMG_R. W,H The check result is generated by determining whether at least one reference pixel that the video encoding circuit 102 of the reference frame IMG_R cannot access is included. This search range is used to find the prediction factor of the current coding unit 204 in the inter-frame prediction mode, and the target prediction factor of the current coding unit 204 is determined based on the check result.

[0043] SRAM can be used to implement the reconstruction / reference buffer (e.g., data buffer 104). Therefore, the size of the data buffer 104 is fixed and independent of the resolution of the input source frame. However, the size of the reference frame (derived from the reconstructed frame) can be larger than the size of the fixed-size reconstruction / reference buffer. To address this problem, the present invention proposes discarding a portion of the reference pixels included in the reference frame (derived from the reconstructed frame) so that not all reference pixels in the reference frame are stored in the fixed-size reconstruction / reference buffer.

[0044] Figure 3 This is a diagram illustrating a second video encoding apparatus according to an embodiment of the present invention. The main difference between video encoding apparatus 300 and video encoding apparatus 100 is that the video encoding circuit 302 of video encoding apparatus 300 includes a dropout circuit (denoted by "DS") 306 connected between the data buffer 304 and the in-loop filter 118. It should be noted that... Figure 3The structure of the video encoding circuit 302 shown is merely illustrative and not intended to limit the invention. In this embodiment, reference pixels of a reference frame are derived from the reconstructed pixels of the input source frame IMG, and the reference pixel data is stored in a data buffer 304, which acts as a reconstruction / reference buffer, wherein the reference pixel data includes information about the pixel values ​​of the reference pixels. The video encoding circuit 302 (specifically, the inter-frame prediction circuit 120) can retrieve a portion of the reference pixel data stored in the data buffer 304 to find the prediction factor of the coding unit in the inter-frame prediction mode.

[0045] In this embodiment, one or more reference pixels of the reference frame (derived from the reconstructed frame) are discarded by the discarding circuit 306 instead of being stored in a data buffer 304 of a fixed size larger than the reference frame. In other words, the reference pixels of the reference frame stored in the data buffer 304 are only a portion of the reference frame. Therefore, the reference frame includes valid reference pixels stored in the data buffer 304 and accessible to the video encoding circuit 302, and also includes invalid reference pixels that are inaccessible to the video encoding circuit 302 due to being discarded by the discarding circuit 306. In this embodiment, the video encoding circuit 302 (specifically, the prediction circuit 119) for... Figure 2 The current coding unit 204 in the second frame IMG2 shown performs prediction to determine the target prediction factor for the current coding unit 204. Since one or more reference pixels not stored in the data buffer 304 are invalid reference pixels, the video coding circuit 302 (specifically, the prediction circuit 119) checks the search range SR against the reference frame IMG_R. W,H The check result is generated by determining whether at least one reference pixel that the video encoding circuit 302 of the reference frame IMG_R cannot access is included. This search range is used to find the prediction factor of the current coding unit 204 in the intra-frame prediction mode, and the target prediction factor of the current coding unit 204 is determined based on the check result.

[0046] Figure 4 This is a diagram illustrating a first example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720. The reference pixels included in the left half of the reference frame IMG_R (i.e., a portion of the reference frame with a resolution of 540x720) are valid reference pixels 402, while the reference pixels included in the right half of the reference frame IMG_R (i.e., another portion of the reference frame with a resolution of 540x720) are invalid reference pixels 404. The reference frame IMG_R is... Figure 1In the case of the video encoding circuit 102 shown, the invalid reference pixel 404 is a reference pixel stored in the data buffer 104 but prevented from being sent to the video encoding circuit 102 due to memory BW limitations. In the reference frame IMG_R generated by... Figure 3 In another case generated by the video encoding circuit 302 shown, the invalid reference pixel 404 is a reference pixel that was not stored in the data buffer 304 because it was discarded by the discarding circuit 306.

[0047] like Figure 4 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 404. Based on the inspection results, the video coding circuits 102 / 302 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 102 / 302 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0048] like Figure 4 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixel 402 and invalid reference pixel 404. Based on the inspection results, the video coding circuits 102 / 302 search for the predictor of coding unit "b" in intra-frame prediction mode by limiting the motion vector (MV) search to only valid reference pixel 402. Figure 5 This is an example in Figure 4 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 5 As shown, the encoding unit 502 to be encoded is in Figure 4 The reference frame IMG_R shown has a search range SR. W,H For example, coding unit 502 could be Figure 4 The encoding unit "b" is shown, and the search range SR W,H It can be Figure 4 The search range SR shown b Within the search range SR W,H There is an invalid reference pixel 504. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 504, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0049] Figure 6This is a diagram illustrating a second example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720. The reference pixels included in the upper half of the reference frame IMG_R (i.e., a portion of the reference frame with a resolution of 1280x360) are valid reference pixels 602, while the reference pixels included in the lower half of the reference frame IMG_R (i.e., another portion of the reference frame with a resolution of 1280x360) are invalid reference pixels 604. The reference frame IMG_R is... Figure 1 In the case of the video encoding circuit 102 shown, the invalid reference pixel 604 is a reference pixel stored in the data buffer 104 but prevented from being sent to the video encoding circuit 102 due to memory BW limitations. In the reference frame IMG_R generated by... Figure 3 In another case generated by the video encoding circuit 302 shown, the invalid reference pixel 604 is a reference pixel that was not stored in the data buffer 304 because it was discarded by the discarding circuit 306.

[0050] like Figure 6 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 604. Based on the inspection results, the video coding circuits 102 / 302 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 102 / 302 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0051] like Figure 6 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixel 602 and invalid reference pixel 604. Based on the inspection results, the video coding circuits 102 / 302 search for the prediction factor of coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixel 602. Figure 7 This is an example in Figure 6 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 7 As shown, the encoding unit 702 to be encoded is in Figure 6 The reference frame IMG_R shown has a search range SR. W,H For example, coding unit 702 could be Figure 6 The encoding unit "b" is shown, and the search range SR W,HIt can be Figure 6 The search range SR shown b Within the search range SR W,H There is an invalid reference pixel 704. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 704, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0052] Figure 8 This is a diagram illustrating a third example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the resolution of the reference frame IMG_R is 1280x720. Reference pixels near the center of the reference frame IMG_R (i.e., a portion of the reference frame with a rectangular shape) are valid reference pixels 802, while reference pixels near the four sides of the reference frame IMG_R (i.e., another portion of the reference frame with a circular shape) are invalid reference pixels 804. The reference frame IMG_R is composed of... Figure 1 In the case of the video encoding circuit 102 shown, the invalid reference pixel 804 is a reference pixel stored in the data buffer 104 but prevented from being sent to the video encoding circuit 102 due to memory BW limitations. In the reference frame IMG_R generated by... Figure 3 In another case generated by the video encoding circuit 302 shown, the invalid reference pixel 804 is a reference pixel that was not stored in the data buffer 304 because it was discarded by the discarding circuit 306.

[0053] like Figure 8 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 804. Based on the inspection results, the video coding circuits 102 / 302 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 102 / 302 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0054] like Figure 8 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixel 802 and invalid reference pixel 804. Based on the inspection results, the video coding circuits 102 / 302 search for the prediction factor of coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixel 802. Figure 9 This is an example in Figure 8The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 9 As shown, the encoding unit 902 to be encoded is in Figure 8 The reference frame IMG_R shown has a search range SR. W,H For example, coding unit 902 could be Figure 8 The encoding unit "b" is shown, and the search range SR W,H It can be Figure 8 The search range SR shown b Within the search range SR W,H There is an invalid reference pixel 904. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 904, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0055] As mentioned above, the data buffer 304 (which acts as a reconstruction / reference buffer) can be implemented using SRAM. Therefore, the size of the data buffer 304 is fixed and independent of the resolution of the input source frame. To preserve more reference pixels accessible to the video encoding circuitry when the size of the reference frame (derived from the reconstructed frame) is larger than the size of the data buffer 304, this invention proposes applying data compression to the reference pixels included in the reference frame.

[0056] Figure 10 This is a diagram illustrating a third video encoding apparatus according to an embodiment of the present invention. The main difference between video encoding apparatus 1000 and video encoding apparatus 300 is that the video encoding circuit 1002 of video encoding apparatus 1000 includes a compression circuit (denoted by "C") 1004, a discard circuit (denoted by "DS") 1006, and a decompression circuit (denoted by "D") 1008. It should be noted that... Figure 10The structure of the video encoding circuit 1002 shown is merely illustrative and not intended to limit the invention. In this embodiment, reference pixels of a reference frame are derived from the reconstructed pixels of the input source frame IMG, and the reference pixel data is stored in a data buffer 304, which acts as a reconstruction / reference buffer. The reference pixel data includes information about the pixel values ​​of the reference pixels. Since the compression circuit 1004 is located between the in-loop filter 118 and the data buffer 304, the reference pixel data in the data buffer 304 is stored in a compressed format. Specifically, the compression circuit 1004 generates an encoded bitstream of the reference pixel data by applying data compression to the pixel values ​​of the reference pixels, and stores the encoded bitstream of the reference pixel data in the data buffer 304. For example, the compression circuit 1004 employs a lossless compression scheme. The decompression circuit 1008 can retrieve a portion of the encoded bitstream of the reference pixel data from the data buffer 304, and can obtain a portion of the reference pixel data by applying data decompression to said portion of the encoded bitstream of the reference pixel data. Therefore, the video coding circuit 1002 (in particular, the inter-frame prediction circuit 120) can refer to the portion of the reference pixel data provided from the decompression circuit 1008 to find the prediction factor of the coding unit in the inter-frame prediction mode.

[0057] However, the size of the compressed reference frame may still be larger than the size of the fixed-size reconstruction / reference buffer. In this implementation, the encoded bitstream of one or more reference pixels of the reference frame (derived from the reconstructed frame) is discarded by the discard circuit 1006 instead of being stored in a fixed-size data buffer 304 that is larger than the size of the compressed reference frame. In other words, the reference pixels of the reference frame that are compressed and then stored in the data buffer 304 are only a portion of the reference frame. Therefore, the reference frame includes valid reference pixels that are stored in the data buffer 104 after compression and can be accessed by the video encoding circuit 1002 via the decompression circuit 1008, and also includes invalid reference pixels that are inaccessible to the video encoding circuit 1002 after compression due to being discarded by the discard circuit 1006.

[0058] Figure 11 This is a diagram illustrating a first example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this example, the discard circuit 1006 is configured to operate based on rows of encoded units. Figure 11As shown, a coding unit line CU_R of a reference frame (derived from the reconstructed frame) comprises multiple coding units A1 to A12. Compression circuit 1004 applies data compression (e.g., lossless compression) to the coding unit line CU_R to generate an encoded bitstream BS_R of the coding unit line CU_R. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_R is the encoded bitstream BS_1 of coding unit group GP_1, consisting of coding units A1 to A8, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_R is the encoded bitstream BS_2 of coding unit group GP_2, consisting of coding units A9 to A12. In this example, discarding circuit 1006 discards the second half (i.e., the last 50% of the size) of the encoded bitstream BS_R so that the encoded bitstream BS_2 of coding unit group GP_2, consisting of coding units A9 to A12, is not stored in data buffer 304. Since the encoded bitstream BS_1 is available in the data buffer 304, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 304 and recover the coding unit group GP_1 (which consists of coding units A1 to A8) by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in coding units A1 to A8 are valid reference pixels accessible to the video encoding circuit 1002, while the reference pixels included in coding units A9 to A12 are invalid reference pixels, which are discarded after compression and are therefore inaccessible to the video encoding circuit 1002.

[0059] In this embodiment, the video encoding circuit 1002 (specifically, the prediction circuit 119) for Figure 2 The current coding unit 204 in the second frame IMG2 shown performs prediction to determine the target prediction factor for the current coding unit 204. Since one or more reference pixels that were not stored in the data buffer 304 after compression are invalid reference pixels, the video coding circuit 1002 (specifically, the prediction circuit 119) checks the search range SR for the reference frame IMG_R. W,H The check result is generated by determining whether at least one reference pixel of the reference frame IMG_R that cannot be accessed by the video coding circuit 1002 is included. This search range is used to find the predictor of the current coding unit 204 in the intra-frame prediction mode, and the target predictor of the current coding unit 204 is determined based on the check result.

[0060] To reduce DRAM bandwidth and / or power consumption, SRAM can be used to implement the rebuild / reference buffer. However, memory bandwidth limitations can still be applied to rebuild / reference buffers implemented with SRAM. Figure 12This is a diagram illustrating a fourth video encoding apparatus according to an embodiment of the present invention. The main difference between video encoding apparatus 1200 and video encoding apparatus 100 is that the video encoding circuit 1202 of video encoding apparatus 1200 includes a compression circuit (denoted by "C") 1004 and a decompression circuit (denoted by "D") 1008. It should be noted that... Figure 12 The structure of the video encoding circuit 1202 shown is merely illustrative and not intended to limit the invention. In this embodiment, reference pixels of a reference frame are derived from the reconstructed pixels of the input source frame IMG, and the reference pixel data is stored in a data buffer 104, which acts as a reconstruction / reference buffer. The reference pixel data includes information about the pixel values ​​of the reference pixels. Since the compression circuit 1004 is located between the in-loop filter 118 and the data buffer 104, the reference pixel data in the data buffer 104 is stored in a compressed format. Specifically, the compression circuit 1004 generates an encoded bitstream of the reference pixel data by applying data compression to the pixel values ​​of the reference pixels, and stores the encoded bitstream of the reference pixel data in the data buffer 104. For example, the compression circuit 1004 employs a lossless compression scheme. The decompression circuit 1008 can retrieve a portion of the encoded bitstream of the reference pixel data from the data buffer 104, and can obtain a portion of the reference pixel data by applying data decompression to said portion of the encoded bitstream of the reference pixel data. Therefore, the video coding circuit 1202 (in particular, the inter-frame prediction circuit 120) can refer to this portion of the reference pixel data provided from the decompression circuit 1008 to find the prediction factor of the coding unit in the inter-frame prediction mode.

[0061] In this embodiment, the memory BW limit is still applied to the data buffer 104 implemented by SRAM. According to the memory BW limit, the encoded bitstream of one or more reference pixels stored in the data buffer 104 can be prevented from being read from the data buffer 104 and sent to the video encoding circuit 102 (specifically, the motion estimation circuit 121 of the prediction circuit 119). In other words, all reference pixels of a reference frame (derived from the reconstructed frame) that is not compressed and subsequently stored in the data buffer 104 are accessible to the video encoding circuit 1202. Therefore, the reference frame includes valid reference pixels stored in the data buffer 104 after compression and accessible to the video encoding circuit 1202 via the decompression circuit 1008, and also includes invalid reference pixels stored in the data buffer 104 after compression and inaccessible to the video encoding circuit 1202 via the decompression circuit 1008 due to the memory BW limit.

[0062] Figure 13This diagram illustrates a second example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this embodiment, memory BW limits are applied based on the row of encoded units. Figure 13 As shown, a coding unit line CU_R of a reference frame (derived from the reconstructed frame) comprises multiple coding units A1 to A12. Compression circuit 1004 applies data compression (e.g., lossless compression) to the coding unit line CU_R to generate an encoded bitstream BS_R of the coding unit line CU_R. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_R is the encoded bitstream BS_1 of coding unit group GP_1 consisting of coding units A1 to A8, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_R is the encoded bitstream BS_2 of coding unit group GP_2 consisting of coding units A9 to A12. In this example, the entire encoded bitstream BS_R is stored in data buffer 104. A memory BW limit is applied to data buffer 104 to prevent the second half (i.e., the last 50% of the size) of the encoded bitstream BS_R from being read from data buffer 104 and sent to video encoding circuit 1202. Since the encoded bitstream BS_1 is available in the data buffer 104, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 104 and recover the encoding unit group GP_1 (which consists of encoding units A1 to A8) by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in the encoding units A1 to A8 are valid reference pixels accessible to the video encoding circuit 1202, while the reference pixels included in the encoding units A9 to A12 are invalid reference pixels, which are inaccessible to the video encoding circuit 1202 due to memory BW limitations.

[0063] In this embodiment, the video encoding circuit 1202 (specifically, the prediction circuit 119) for Figure 2 The current coding unit 204 in the second frame IMG2 shown performs prediction to determine the target prediction factor for the current coding unit 204. Since one or more reference pixels stored in the data buffer 104 after compression may be invalid reference pixels, the video coding circuit 1202 (specifically, the prediction circuit 119) checks the search range SR for the reference frame IMG_R. W,H The check result is generated by determining whether at least one reference pixel of the reference frame IMG_R that cannot be accessed by the video coding circuit 1202 is included. This search range is used to find the prediction factor of the current coding unit 204 in the intra-frame prediction mode, and the target prediction factor of the current coding unit 204 is determined based on the check result.

[0064] Figure 14 This is a diagram illustrating a fourth example of performing prediction on coding units to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720 and includes valid reference pixels 1402 and invalid reference pixels 1404. Since the discard circuit 1006 operates on a row-by-row basis, and / or the memory BW limit is applied to the data buffer 104 on a row-by-row basis, the number of invalid reference pixels in each row of coding units is not fixed due to the fact that the lossless compressed bitstream of each row of coding units does not have a fixed size. In the reference frame IMG_R, which is... Figure 12 In the case of the video encoding circuit 1202 shown, the invalid reference pixel is a reference pixel that is stored in the data buffer 104 after compression but is prevented from being sent to the video encoding circuit 1202 due to memory BW limitations. In the reference frame IMG_R, which is generated by... Figure 10 In another case generated by the video encoding circuit 1002 shown, the invalid reference pixel is a reference pixel that was not stored in the data buffer 304 after being discarded by the discarding circuit 1006 after compression.

[0065] like Figure 14 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 1404. Based on the inspection results, the video coding circuits 1002 / 1202 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 1002 / 1202 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0066] like Figure 14 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixel 1402 and invalid reference pixel 1404. Based on the inspection results, the video encoding circuits 1002 / 1202 search for the prediction factor of the coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixel 1402. Figure 15 This is an example in Figure 14 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 15 As shown, the encoding unit 1502 to be encoded is in Figure 14 The reference frame IMG_R shown has a search range SR.W,H Within the search range SR W,H There is an invalid reference pixel 1504. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 1504, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0067] like Figure 11 As shown, the discard circuit 1006 operates based on the coded unit row. For example... Figure 13 As shown, the memory BW is limited to the data buffer 104 based on the row of encoded units. However, these are merely illustrative and not intended to limit the invention.

[0068] In an alternative design, the discard circuit 1006 may be configured to operate on a frame-based basis, and / or the memory BW may be restricted to a frame-based basis and applied to the data buffer 104.

[0069] Figure 16 This is a diagram illustrating a third example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this example, the discard circuit 1006 is configured to operate on a frame-based basis. Figure 16As shown, the reference frame IMG_R is derived from the reconstructed frame and includes multiple coding units. Compression circuit 1004 applies data compression (e.g., lossless compression) to the reference frame IMG_R to generate an encoded bitstream BS_IMG of the reference frame IMG_R. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_IMG is encoded bitstream BS_1, consisting of coding unit group GP_1 composed of some coding units from the reference frame IMG_R, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_IMG is encoded bitstream BS_2, consisting of the remaining coding units from the reference frame IMG_R, consisting of coding unit group GP_2. In this example, discarding circuit 1006 discards the second half (i.e., the last 50% of the size) of the encoded bitstream BS_IMG so that the encoded bitstream BS_2 of coding unit group GP_2 is not stored in data buffer 304. Since the encoded bitstream BS_1 is available in the data buffer 304, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 304 and recover the encoding unit group GP_1 by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in the encoding units of the encoding unit group GP_1 are valid reference pixels accessible to the video encoding circuit 1002, while the reference pixels included in the encoding units of the encoding unit group GP_2 are invalid reference pixels, which are discarded after compression and are therefore inaccessible to the video encoding circuit 1002.

[0070] Figure 17 This is a diagram illustrating a fourth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this example, memory BW limits are applied on a frame basis. Figure 17As shown, the reference frame IMG_R is derived from the reconstructed frame and includes multiple coding units. Compression circuit 1004 applies data compression (e.g., lossless compression) to the reference frame IMG_R to generate an encoded bitstream BS_IMG of the reference frame IMG_R. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_IMG is encoded bitstream BS_1, consisting of coding unit group GP_1 composed of some coding units from the reference frame IMG_R, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_IMG is encoded bitstream BS_2, consisting of coding unit group GP_2 composed of the remaining coding units from the reference frame IMG_R. The entire encoded bitstream BS_IMG is stored in data buffer 104. A memory BW limit is applied to data buffer 104 to prevent the reading of the last half (i.e., the last 50% of the size) of the encoded bitstream BS_IMG from data buffer 104 and its transmission to video encoding circuit 1202. Since the encoded bitstream BS_1 is available in the data buffer 104, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 104 and recover the encoding unit group GP_1 by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in the encoding units of the encoding unit group GP_1 are valid reference pixels accessible to the video encoding circuit 1202, while the reference pixels included in the encoding units of the encoding unit group GP_2 are invalid reference pixels, which are inaccessible to the video encoding circuit 1202 due to memory BW limitations.

[0071] Figure 18 This is a diagram illustrating a fifth example of performing prediction on coding units to determine target prediction factors for the coding units according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720 and includes valid reference pixels 1802 and invalid reference pixels 1804. Since the discard circuit 1006 operates on a frame-based basis, and / or the memory BW limit is applied to the data buffer 104 on a frame-based basis, the reference frame IMG_R can have at most one row of coding units, which includes at least one valid reference pixel 1802 and at least one invalid reference pixel 1804. The reference frame IMG_R is... Figure 12 In the case of the video encoding circuit 1202 shown, the invalid reference pixel 1804 is a reference pixel that is stored in the data buffer 104 after compression but is prevented from being sent to the video encoding circuit 1202 due to memory BW limitations. In the reference frame IMG_R, which is generated by... Figure 10In another case generated by the video encoding circuit 1002 shown, the invalid reference pixel 1804 is a reference pixel that was not stored in the data buffer 304 after being discarded by the discarding circuit 1006 after compression.

[0072] like Figure 18 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 1804. Based on the inspection results, the video coding circuits 1002 / 1202 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 1002 / 1202 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0073] like Figure 18 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixel 1402 and invalid reference pixel 1404. Based on the inspection results, the video encoding circuits 1002 / 1202 search for the prediction factor of the coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixel 1802. Figure 19 This is an example in Figure 18 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 19 As shown, the encoding unit 1902 to be encoded is in Figure 18 The reference frame IMG_R shown has a search range SR. W,H For example, coding unit 1902 could be Figure 18 The encoding unit "b" is shown, and the search range SR W,H It can be Figure 18 The search range SR shown b Within the search range SR W,H There is an invalid reference pixel 1904. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 1904, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0074] In another alternative design, the discard circuit 1006 can be configured to operate on a pixel-row basis, and / or the memory BW can be restricted to a pixel-row basis and applied to the data buffer 104.

[0075] Figure 20This is a diagram illustrating a fifth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this example, the discard circuit 1006 is configured to operate on a pixel-row basis. Figure 16 As shown, a pixel row P_L of a reference frame (derived from the reconstructed frame) comprises multiple pixels. Compression circuit 1004 applies data compression (e.g., lossless compression) to the pixel row P_L to generate an encoded bitstream BS_L of the pixel row P_L. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_L is the encoded bitstream BS_1 of pixel group PG_1, consisting of some pixels from pixel row P_L, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_L is the encoded bitstream BS_2 of pixel group PG_2, consisting of the remaining pixels from pixel row P_L. In this example, discarding circuit 1006 discards the second half (i.e., the last 50% of the size) of the encoded bitstream BS_L so that the encoded bitstream BS_2 of pixel group PG_2 is not stored in data buffer 304. Since the encoded bitstream BS_1 is available in the data buffer 304, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 304 and recover the pixel group PG_1 by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in the pixel group PG_1 are valid reference pixels accessible to the video encoding circuit 1002, while the reference pixels included in the pixel group PG_2 are invalid reference pixels, which are discarded after compression and are therefore inaccessible to the video encoding circuit 1002.

[0076] Figure 21 This is a diagram illustrating a sixth example of accessing a data buffer via compression / decompression according to an embodiment of the present invention. In this example, memory BW limits are applied on a pixel-row basis. Figure 21As shown, a pixel row P_L of a reference frame (derived from the reconstructed frame) comprises multiple pixels. Compression circuit 1004 applies data compression (e.g., lossless compression) to the pixel row P_L to generate an encoded bitstream BS_L of the pixel row P_L. The first half (i.e., the first 50% of the size) of the encoded bitstream BS_L is the encoded bitstream BS_1 of pixel group PG_1, consisting of some pixels from pixel row P_L, while the second half (i.e., the last 50% of the size) of the encoded bitstream BS_L is the encoded bitstream BS_2 of pixel group PG_2, consisting of the remaining pixels from pixel row P_L. In this example, the entire encoded bitstream BS_L is stored in data buffer 104. A memory BW limit is applied to data buffer 104 to prevent the second half (i.e., the last 50% of the size) of the encoded bitstream BS_L from being read from data buffer 104 and sent to video encoding circuit 1202. Since the encoded bitstream BS_1 is available in the data buffer 104, the decompression circuit 1008 can read the encoded bitstream BS_1 from the data buffer 104 and recover the pixel group PG_1 by applying data decompression to the encoded bitstream BS_1. Therefore, the reference pixels included in the pixel group PG_1 are valid reference pixels accessible to the video encoding circuit 1202, while the reference pixels included in the pixel value PG_2 are invalid reference pixels, which are inaccessible to the video encoding circuit 1202 due to memory BW limitations.

[0077] Figure 22 This is a diagram illustrating a sixth example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720 and includes valid reference pixels 2202 and invalid reference pixels 2204. Since the discarding circuit 1006 operates on a pixel-row basis, and / or the memory BW limit is applied to the data buffer 104 on a pixel-row basis, the number of invalid reference pixels in each pixel row is not fixed due to the fact that the lossless compressed bitstream of each pixel row does not have a fixed size. In the reference frame IMG_R, which is... Figure 12 In the case of the video encoding circuit 1202 shown, the invalid reference pixel 2204 is a reference pixel that is stored in the data buffer 104 after compression but is prevented from being sent to the video encoding circuit 1202 due to memory BW limitations. In the reference frame IMG_R, which is generated by... Figure 10 In another case generated by the video encoding circuit 1002 shown, the invalid reference pixel 2204 is a reference pixel that was not stored in the data buffer 304 after being discarded by the discarding circuit 1006 after compression.

[0078] like Figure 22 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels 2204. Based on the inspection results, the video coding circuits 1002 / 1202 force the derivation of the target prediction factor of coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuits 1002 / 1202 force the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0079] like Figure 22 As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixels 2202 and invalid reference pixels 2204. Based on the inspection results, the video encoding circuits 1002 / 1202 search for the prediction factor of the coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixels 2202. Figure 23 This is an example in Figure 22 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 23 As shown, the encoding unit 2302 to be encoded is in Figure 22 The reference frame IMG_R shown has a search range SR. W,H Within the search range SR W,H There is an invalid reference pixel 2304. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixel 2304, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0080] As mentioned above, the data buffer 304 can be implemented using SRAM (this data buffer acts as a reconstruction / reference buffer). Therefore, the size of the data buffer 304 is fixed and independent of the resolution of the input source frame. The invention also proposes applying pre-discarding before compression. This allows the data buffer 304 to have a smaller size.

[0081] Figure 24 This is a diagram illustrating a fifth video encoding apparatus according to an embodiment of the present invention. The main difference between video encoding apparatus 2400 and video encoding apparatus 1000 is that the video encoding circuit 2402 of video encoding apparatus 2400 includes a pre-drop circuit (denoted as "PDS") 2420 connected between the in-loop filter 118 and the compression circuit 1004. It should be noted that... Figure 24The structure of the video encoding circuit 2402 shown is merely illustrative and not intended to limit the invention. Reference pixels for a reference frame are derived from the reconstructed pixels of the input source frame IMG. In this embodiment, the pre-drop circuit 2420 is configured to discard reference pixels that are part of the reference frame and output the remaining reference pixels belonging to the reference frame to the compression circuit 1004.

[0082] Reference pixel data is stored in data buffer 304, which acts as a reconstruction / reference buffer. This reference pixel data includes information about the pixel values ​​of reference pixels belonging to the remainder of the reference frame (i.e., reference pixels in the reference frame that were not discarded by the pre-discarding circuit 2420). Since compression circuit 1004 is located between pre-discarding circuit 2420 and data buffer 304, the reference pixel data in data buffer 304 is stored in a compressed format. Specifically, compression circuit 1004 generates an encoded bitstream of reference pixel data by applying data compression to the pixel values ​​of reference pixels belonging to the remainder of the reference frame, and stores this encoded bitstream in data buffer 304. For example, compression circuit 1004 employs a lossless compression scheme. Decompression circuit 1008 can retrieve a portion of the encoded bitstream of reference pixel data from data buffer 304, and can obtain a portion of the reference pixel data by applying data decompression to that portion of the encoded bitstream of reference pixel data. Therefore, the video coding circuit 2402 (in particular, the inter-frame prediction circuit 120) can refer to the portion of the reference pixel data provided from the decompression circuit 1008 to find the prediction factor of the coding unit in the inter-frame prediction mode.

[0083] Because reference pixels in the reference frame are pre-discarded, compression circuit 1004 applies data compression (e.g., lossless compression) to a portion of the reference frame. However, the size of the compressed portion of the reference frame may still be larger than the size of data buffer 304. In this embodiment, the encoded bitstream of one or more reference pixels of the portion of the reference frame is discarded by discarding circuit 1006 instead of being stored in data buffer 304, which has a fixed size larger than the compressed portion of the reference frame. In other words, the reference pixels of the portion of the reference frame that are compressed and then stored in data buffer 304 are only a portion of the portion of the reference frame. Therefore, the reference frame includes valid reference pixels that are stored in data buffer 304 after pre-discarding and compression and that can be accessed by video encoding circuit 1002 via decompression circuit 1008, and also includes invalid reference pixels that are inaccessible to video encoding circuit 1002 before compression due to pre-discarding by discarding circuit 2420 or after compression due to discarding by discarding circuit 1006.

[0084] In this embodiment, the video encoding circuit 2402 (specifically, the prediction circuit 119) for Figure 2 The current coding unit 204 in the second frame IMG2 shown performs prediction to determine the target prediction factor for the current coding unit 204. Since one or more reference pixels of this portion of the reference frame that were not stored in the data buffer 304 after compression are invalid reference pixels, the video coding circuit 2402 (specifically, the prediction circuit 119) checks the search range SR for the reference frame IMG_R. W,H The check result is generated by determining whether at least one reference pixel of the reference frame IMG_R that cannot be accessed by the video coding circuit 2402 is included. This search range is used to find the prediction factor of the current coding unit 204 in the intra-frame prediction mode, and the target prediction factor of the current coding unit 204 is determined based on the check result.

[0085] Figure 25 This is a diagram illustrating a seventh example of performing prediction on a coding unit to determine a target prediction factor for the coding unit according to an embodiment of the present invention. In this example, the reference frame IMG_R has a resolution of 1280x720 and includes valid reference pixels 2502 and invalid reference pixels 2504 and 2506. The reference frame IMG_R is... Figure 24 In the case generated by the video encoding circuit 2402 shown, invalid reference pixel 2506 is a reference pixel that was not received by the compression circuit 1004 due to being pre-discarded by the pre-discarding circuit 2420, and is a reference pixel that was not stored in the data buffer 304 after compression due to being discarded by the discarding circuit 1006. In this example, the discarding circuit 1006 operates on a pixel row basis. Therefore, the number of invalid reference pixels in each pixel row is not fixed because the lossless compressed bitstream of each pixel row does not have a fixed size.

[0086] like Figure 25 As shown, all reference pixels within the search range SRa of the coding unit marked "a" are invalid reference pixels. This includes invalid reference pixels 2504 and 2506. Based on the inspection results, the video coding circuit 2402 forces the derivation of the target prediction factor for coding unit "a" from the previously reconstructed pixels in the current frame being encoded. Assuming coding unit "a" is... Figure 2 As shown in the current coding unit 204, the video coding circuit 2402 forces the target prediction factor of coding unit "a" to be derived based on the previously reconstructed pixels 208. For example, the coding mode of coding unit "a" is forced to be intra-frame prediction mode, palette mode, or intra-block copy mode.

[0087] like Figure 25As shown, the search range SR of the coding unit marked with "b" b The reference pixels within the frame include valid reference pixels 2502 and invalid reference pixels 2504. Based on the inspection results, the video encoding circuit 1202 searches for the prediction factor of the coding unit "b" in intra-frame prediction mode by limiting the MV search to only valid reference pixels 2502. Figure 26 This is an example in Figure 25 The diagram shows an example of an MV search that restricts the range of coding units on the reference frame IMG_R. (See diagram for example.) Figure 26 As shown, the encoding unit 2602 to be encoded is in Figure 25 The reference frame IMG_R shown has a search range SR. W,H Within the search range SR W,H There are invalid reference pixels 2604 and 2606 in memory. For example, invalid reference pixel 2604 could be... Figure 25 The invalid reference pixel 2504 shown is a portion of the invalid reference pixel 2606, while the invalid reference pixel 2606 could be... Figure 25 The example shown is a subset of invalid reference pixels 2506. Therefore, if the derivation of the corresponding predictor includes any invalid reference pixels 2604 / 2606, the motion vector is invalid. Conversely, if all pixels used to derive the corresponding predictor are valid reference pixels, the motion vector is valid.

[0088] Above Figure 4 , Figure 6 as well as Figure 8 In the example shown, the video encoding circuitry cannot access 50% of the reference frame. Above... Figure 11 In the example shown, the video encoding circuit cannot access 50% of the encoded bitstreams 11, 13, 16, 17, 20, and 21, which constitute 50% of the encoded bitstream. However, these are merely illustrative and not intended to limit the invention. In practice, the drop threshold can be set to a different percentage value than 50%. That is, the drop threshold can be adjusted according to actual implementation considerations.

[0089] Those skilled in the art will readily observe that various modifications and alterations can be made to the apparatus and method while maintaining the teachings of the present invention. Therefore, the above disclosure should be interpreted as being limited only by the scope of the appended claims.

Claims

1. A video encoding apparatus comprising: a data buffer; and a video encoding circuit configured to encode a plurality of frames into a bitstream, wherein each frame comprises a plurality of coding units, each coding unit comprising a plurality of pixels, the frames comprising a first frame and a second frame, the encoding operation of the first frame comprising: deriving a plurality of reference pixels of a reference frame from a plurality of reconstructed pixels of the first frame, respectively; and storing reference pixel data into the data buffer for inter prediction, wherein the reference pixel data comprises information of pixel values of the reference pixels; and the encoding operation of the second frame comprising: performing prediction on a coding unit in the second frame to determine a target predictor of the coding unit, including a step of: determining the target predictor of the coding unit according to whether a search range for searching a predictor of the coding unit for the reference frame includes at least one reference pixel of the reference frame that is discarded before encoding or after encoding such that the at least one reference pixel is not stored in the data buffer or prevented from being read from the data buffer and sent to the video encoding circuit as pre-encoding data or post-encoding data of the at least one reference pixel. the video encoding circuit is configured to store pixel values of the reference pixels directly into the data buffer to serve as the reference pixel data.

2. The video coding device of claim 1, wherein, a size of the data buffer is smaller than a size of the reference frame, the reference pixels are only a portion of the reference frame, and the video encoding circuit comprises:

3. The video coding device of claim 2, wherein, a discarding circuit configured to discard the at least one reference pixel of the reference frame. the at least one reference pixel is a portion of the reference pixels; and is prevented from being read from the data buffer and sent to the video encoding circuit according to a bandwidth limit applied to the data buffer.

4. The video coding device of claim 2, wherein, the video encoding circuit comprises:

5. The video coding device of claim 1, wherein, a compression circuit configured to generate an encoded bitstream of the reference pixel data by applying data compression to pixel values of the reference pixels, wherein the encoded bitstream of the reference pixel data is stored into the data buffer. a lossless compression scheme is employed by the data compression.

6. The video coding device of claim 5, wherein, a size of the data buffer is smaller than a size of a compressed reference frame derived by applying the data compression to the reference frame, the reference pixels are only a portion of the reference frame, and the video encoding circuit comprises:

7. The video coding device of claim 5, wherein, a discarding circuit configured to discard the encoded bitstream of the at least one reference pixel of the reference frame. the at least one reference pixel is a portion of the reference pixels; and the encoded bitstream of the at least one reference pixel is prevented from being read from the data buffer and sent to the video encoding circuit according to a bandwidth limit applied to the data buffer.

8. The video coding device of claim 5, wherein, the reference pixels are a portion of the reference frame, and the video encoding circuit further comprises:

9. The video coding device of claim 5, wherein, a discarding circuit configured to discard the at least one reference pixel of the reference frame. a pre-discarding circuit configured to discard reference pixels belonging to another portion of the reference frame so that reference pixels belonging to the another portion of the reference frame are not fed into the compression circuit.

10. The video coding device of claim 9, wherein, The at least one reference pixel is a portion of the reference pixels belonging to the another portion of the reference frame.

11. The video coding device of claim 1, wherein, When the video encoding circuit has access to a portion of reference pixels within the search range and does not have access to another portion of reference pixels within the search range, and the video encoding circuit is forced to derive the target predictor of the coding unit from previously reconstructed pixels in the second frame.

12. The video coding device of claim 1, wherein, When the video encoding circuit has access to a portion of reference pixels within the search range and does not have access to another portion of reference pixels within the search range, and the video encoding circuit is forced to derive the target predictor of the coding unit from previously reconstructed pixels in the second frame.

13. A video encoding method of encoding a plurality of frames into a bitstream, each frame comprising a plurality of coding units, each coding unit comprising a plurality of pixels, the video encoding method comprising the steps of: encoding a first frame comprised in the frames, comprising the steps of: deriving a plurality of reference pixels of a reference frame from a plurality of reconstructed pixels of the first frame, respectively; and storing reference pixel data into a data buffer for inter prediction, wherein the reference pixel data comprises information of pixel values of the reference pixels; and encoding a second frame comprised in the frames, comprising the steps of: performing prediction on a coding unit in the second frame to determine a target predictor of the coding unit, comprising the steps of: determining the target predictor of the coding unit according to whether a search range for searching a predictor of the coding unit for the reference frame comprises at least one reference pixel of the reference frame that is inaccessible by a video encoding circuit, wherein the at least one reference pixel of the reference frame is discarded before encoding or after encoding so that the at least one reference pixel is not stored in the data buffer or prevented from being read from the data buffer and sent to the video encoding circuit with pre-encoding data or post-encoding data of the at least one reference pixel.

14. The video coding method of claim 13, wherein, storing pixel values of the reference pixels directly into the data buffer to serve as the reference pixel data.

15. The video coding method of claim 13, wherein, The at least one reference pixel is a portion of the reference pixels, and the step of encoding the second frame comprised in the frames further comprises: preventing the at least one reference pixel from being read from the data buffer for sending according to a bandwidth limit applied to the data buffer.

16. The video coding method of claim 13, wherein, The step of storing the reference pixel data into the data buffer for inter prediction comprises: generating an encoded bitstream of the reference pixel data by applying data compression to pixel values of the reference pixels; and storing the encoded bitstream of the reference pixel data into the data buffer.

17. The video coding method of claim 13, wherein, The data compression employs a lossless compression scheme.

18. The video coding method of claim 13, wherein, The at least one reference pixel is a part of the reference pixel, and the step of encoding the second frame comprised in the frame further comprises: Preventing reading of the encoded bitstream of the at least one reference pixel from the data buffer for transmission, according to a bandwidth limit applied to the data buffer.

19. The video coding method of claim 13, wherein, The reference pixel is a part of the reference frame, and the step of encoding the first frame comprised in the frame further comprises: Discarding reference pixels belonging to another part of the reference frame so that reference pixels belonging to the another part of the reference frame are not processed by data compression.

20. The video encoding method of claim 19, wherein, The at least one reference pixel is a part of the reference pixel belonging to the another part of the reference frame.

21. The video encoding method of claim 13, wherein, When the video encoding circuit has access to all reference pixels within the search range, the step of determining the target prediction factor of the coding unit comprises: Forcing derivation of the target prediction factor of the coding unit from previously reconstructed pixels in the second frame.

22. The video encoding method of claim 13, wherein, When the video encoding circuit has access to a part of reference pixels within the search range, and the video encoding circuit has no access to another part of reference pixels within the search range, the step of determining the target prediction factor of the coding unit comprises: Finding the prediction factor of the coding unit in the inter prediction mode by limiting the motion vector search to only the part of reference pixels within the search range.

Images