Video processing methods and devices

Abstract

This disclosure relates to a video processing method and apparatus. The video processing method includes: acquiring coding tree units of image frames to be encoded in a target video; for each coding tree unit, performing the following predetermined processing: acquiring a first pixel set and a second pixel set of the current coding tree unit, wherein the first pixel set includes pixels in the bottom four rows of the current coding tree unit before deblocking filtering, and the second pixel set is acquired after deblocking filtering is performed on the current coding tree unit and the right-side adjacent coding tree units of the current coding tree unit, and the second pixel set includes pixels in the current coding tree unit excluding the bottom four rows after deblocking filtering; performing information statistics on the first pixel set and the second pixel set, and determining an adaptive sample compensation mode for the current coding tree unit based on the information statistics results; and writing relevant information of the adaptive sample compensation mode into the bitstream to be sent to the decoding end.

Description

Technical Field

[0001] This disclosure relates to the field of video processing, and more particularly to a video processing method and apparatus. Background Technology

[0002] High Efficiency Video Coding (HEVC) introduces Sample Adaptive Offset (SAO) technology to mitigate ringing effects. The overall SAO process is as follows: information statistics -> SAO mode for the luma coding tree block (CTB) -> SAO mode for the chroma CTB -> optimal SAO mode for the coding tree unit (CTU) -> SAO filtering for the CTU. SAO uses the CTU as the basic unit. The information statistics part requires the reconstructed values ​​of the current CTU for statistical analysis. However, due to the effects of multi-threaded parallelism and deblocking, the rightmost and bottom four rows and four columns of pixels in the current CTU are not the final reconstructed pixels. Therefore, the current SAO implementation skips the rightmost and bottommost pixels of each CTU. This omission of some pixels leads to inaccurate SAO information statistics, resulting in suboptimal parameter decisions. Currently, the common solution is to use the pixels before deblocking for the skipped pixels. Although this alleviates the problem of directly skipping some pixels, the pixel distribution before deblocking is different from that after deblocking. Therefore, using the pixels before deblocking for the skipped pixels will still lead to inaccurate information statistics. Summary of the Invention

[0003] This disclosure provides a video processing method and apparatus to at least solve the problem of inaccurate information statistics in related technologies.

[0004] According to a first aspect of the present disclosure, a video processing method is provided, comprising: acquiring coding tree units of image frames to be encoded in a target video; for each coding tree unit, performing the following predetermined processing: acquiring a first pixel set and a second pixel set of the current coding tree unit, wherein the first pixel set includes pixels in the bottom four rows of the current coding tree unit before deblocking filtering, and the second pixel set is acquired after deblocking filtering is performed on the current coding tree unit and the right-side adjacent coding tree units of the current coding tree unit, and the second pixel set includes pixels in the current coding tree unit excluding the bottom four rows after deblocking filtering; performing information statistics on the first pixel set and the second pixel set, and determining an adaptive sample compensation mode for the current coding tree unit based on the information statistics results; writing relevant information of the adaptive sample compensation mode into a bitstream to be sent to a decoding end, wherein the decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

[0005] Optionally, before performing the predetermined processing, the method further includes: determining whether to enable adaptive sample compensation based on relevant information of the image frame to be encoded; and performing the predetermined processing if it is determined that adaptive sample compensation is enabled.

[0006] Optionally, the relevant information includes the frame type of the image frame to be encoded and / or a preset quality coefficient, wherein the preset quality coefficient represents a pre-set coefficient used to indicate the encoding quality of the target video; based on the relevant information of the image frame to be encoded, determining whether to enable adaptive sample compensation includes: determining to enable adaptive sample compensation if one of the following conditions is met: the preset quality coefficient is less than a first preset value; the preset quality coefficient is greater than or equal to the first preset value and less than a second preset value, and the frame type of the image frame to be encoded is not a non-reference B-frame; the preset quality coefficient is greater than or equal to the second preset value, and the frame type of the image frame to be encoded is not a B-frame.

[0007] Optionally, the relevant information includes the prediction mode of the coding tree unit in the image frame to be encoded. Based on the relevant information of the image frame to be encoded, it is determined whether to enable adaptive sample compensation, including: determining the number of coding tree units in the image frame to be encoded whose prediction mode adopts the skip mode based on the identifier of the prediction mode in the image frame to be encoded; and determining to enable adaptive sample compensation if the number is less than a third preset value.

[0008] Optionally, the relevant information includes the rate-distortion cost of the image frame to be encoded. Based on the relevant information of the image frame to be encoded, it is determined whether to enable adaptive sample compensation, including: obtaining the intra-frame rate-distortion cost of the image frame to be encoded, the inter-frame rate-distortion cost of the image frame to be encoded and the corresponding reference frame; and determining to enable adaptive sample compensation if the difference between the inter-frame rate-distortion cost and the intra-frame rate-distortion cost is less than a fourth preset value.

[0009] Optionally, the above method further includes: if it is determined that adaptive sample compensation is not enabled, writing an identifier indicating that adaptive sample compensation is not enabled for the image frame to be encoded into the bitstream to be sent to the decoding end.

[0010] Optionally, the above method further includes: determining the number of coding tree units in the image frame to be encoded that are not compensated based on the adaptive sample compensation mode of each coding tree unit in the determined image frame to be encoded; and writing an identifier indicating that adaptive sample compensation is not enabled in the image frame to be encoded into the bitstream to be sent to the decoding end when the number of coding tree units in the image frame to be encoded exceeds a preset ratio.

[0011] According to a second aspect of the present disclosure, a video processing apparatus is provided, comprising: an acquisition unit configured to acquire coding tree units of image frames to be encoded in a target video; and a processing unit configured to perform the following predetermined processing for each coding tree unit: acquiring a first pixel set and a second pixel set of the current coding tree unit, wherein the first pixel set includes pixels in the bottom four rows of the current coding tree unit before deblocking filtering, and the second pixel set is acquired after deblocking filtering is performed on the current coding tree unit and the right-side adjacent coding tree units of the current coding tree unit, and the second pixel set includes pixels in the current coding tree unit other than the bottom four rows after deblocking filtering; performing information statistics on the first pixel set and the second pixel set, and determining an adaptive sample compensation mode for the current coding tree unit based on the information statistics results; and a writing unit configured to write relevant information of the adaptive sample compensation mode into a bitstream to be sent to a decoding end, wherein the decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

[0012] Optionally, the processing unit is further configured to determine whether to enable adaptive sample compensation based on relevant information of the image frame to be encoded before performing the predetermined processing; if it is determined that adaptive sample compensation is enabled, the predetermined processing is performed.

[0013] Optionally, the relevant information includes the frame type of the image frame to be encoded and / or a preset quality coefficient, wherein the preset quality coefficient represents a pre-set coefficient used to indicate the encoding quality of the target video; the processing unit is further configured to determine to enable adaptive sample compensation if one of the following conditions is met: the preset quality coefficient is less than a first preset value; the preset quality coefficient is greater than or equal to the first preset value and less than a second preset value, and the frame type of the image frame to be encoded is not a non-reference B-frame; the preset quality coefficient is greater than or equal to the second preset value, and the frame type of the image frame to be encoded is not a B-frame.

[0014] Optionally, the relevant information includes the prediction mode of the coding tree unit in the image frame to be encoded. The processing unit is also configured to determine the number of coding tree units in the image frame to be encoded whose prediction mode adopts the skip mode based on the identifier of the prediction mode in the image frame to be encoded; and to determine to enable adaptive sample compensation if the number is less than a third preset value.

[0015] Optionally, the relevant information includes the rate-distortion cost of the image frame to be encoded. The processing unit is also configured to acquire the intra-frame rate-distortion cost of the image frame to be encoded, the inter-frame rate-distortion cost of the image frame to be encoded and the corresponding reference frame; and determine to enable adaptive sampling compensation if the difference between the inter-frame rate-distortion cost and the intra-frame rate-distortion cost is less than a fourth preset value.

[0016] Optionally, the writing unit is also configured to write an identifier indicating that adaptive sample compensation is not enabled into the bitstream to be sent to the decoder if it is determined that adaptive sample compensation is not enabled.

[0017] Optionally, the writing unit is further configured to determine the number of coding tree units in the image frame to be encoded that are not compensated based on the adaptive sample compensation mode of each coding tree unit in the determined image frame to be encoded; if the number of coding tree units in the image frame to be encoded that are not compensated exceeds a preset ratio, an identifier indicating that the image frame to be encoded has not enabled adaptive sample compensation is written into the bitstream to be sent to the decoding end.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a video processing method according to the present disclosure.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes at least one processor to perform the video processing method as described above according to the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement the video processing method according to the present disclosure.

[0021] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0022] According to the video processing method and apparatus of this disclosure, for each coding tree unit of the image frame to be encoded, the first pixels of the bottom four rows of the current coding tree unit before deblocking are obtained. After the deblocking filtering of the current coding tree unit and the coding tree unit adjacent to the right of the current coding tree unit is completed, the first pixels of the current coding tree unit after deblocking filtering, excluding the bottom four rows of pixels, are also obtained. This ensures that this disclosure does not skip some pixels during information statistics, and that only the bottom four rows of pixels used in the information statistics still use the pixels before deblocking filtering, while the other pixels use the pixels after deblocking filtering. This improves the accuracy of information statistics, thereby obtaining a relatively accurate compensation method and achieving better encoding quality. Therefore, this disclosure solves the problem of inaccurate information statistics in related technologies.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0025] Figure 1 This is a schematic diagram illustrating an implementation scenario of a video processing method according to an exemplary embodiment of the present disclosure;

[0026] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment;

[0027] Figure 3 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment;

[0028] Figure 4 This is a block diagram of an electronic device 400 according to an embodiment of the present disclosure. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0032] This disclosure provides a video processing method that can improve the accuracy of information statistics, thereby bringing better encoding quality. The following explanation uses a video transmission scenario as an example.

[0033] Figure 1 This is a schematic diagram illustrating an implementation scenario of a video processing method according to an exemplary embodiment of the present disclosure, such as... Figure 1 The implementation scenario includes server 100, user terminal 110, and user terminal 120. The number of user terminals is not limited to two, and includes, but is not limited to, devices such as mobile phones and personal computers. The user terminals can be equipped with cameras to acquire video. The server can be a single server, a server cluster composed of several servers, or a cloud computing platform or virtualization center.

[0034] User terminal 110 acquires target video through a camera, first dividing each frame of the target video into equal-sized coding tree units, and then encoding them in the encoder in server 100, i.e., acquiring coding tree units of the image frames to be encoded in the target video. For each coding tree unit, user terminal 110 performs the following predetermined processing: acquiring the first pixel set and the second pixel set of the current coding tree unit, wherein the first pixel set includes the pixels in the bottom four rows of the current coding tree unit before deblocking filtering, and the second pixel set is acquired after deblocking filtering is performed on the current coding tree unit and the right-side adjacent coding tree units of the current coding tree unit, and the second pixel set includes the pixels in the current coding tree unit excluding the bottom four rows after deblocking filtering; performing information statistics on the first pixel set and the second pixel set, and determining the adaptive sample compensation mode of the current coding tree unit based on the information statistics results; after acquiring the adaptive sample compensation mode, user terminal 110 writes the relevant information of the adaptive sample compensation mode into the bitstream to be sent to user terminal 120, and user terminal 120 performs sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

[0035] It should be noted that user terminals 110 and 120 can perform this task independently, or they can receive data services from server 100; this disclosure does not limit this. When receiving data services from server 100, user terminal 110 acquires the target video through a camera and sends the target video to server 100. After receiving the target video, server 100 first divides each frame of the target video into equal-sized coding tree units, and then encodes them in the encoder in server 100, that is, it acquires the coding tree units of the image frames to be encoded in the target video. For each coding tree unit, server 100 performs the following predetermined processing: acquiring the first pixel set and the second pixel set of the current coding tree unit, wherein the first pixel set includes the bottom four rows of pixels before deblocking filtering in the current coding tree unit. The first pixel set and the second pixel set are obtained after deblocking filtering is performed on the current coding tree unit and the right-adjacent coding tree units of the current coding tree unit. The second pixel set includes the pixels in the current coding tree unit excluding the bottom four rows after deblocking filtering. Information statistics are performed on the first pixel set and the second pixel set. Based on the information statistics results, the adaptive sample compensation mode of the current coding tree unit is determined. After obtaining the adaptive sample compensation mode, the server 100 writes the relevant information of the adaptive sample compensation mode into the bitstream to be sent to the user terminal 120. The user terminal 120 performs sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

[0036] The video processing method and apparatus according to exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0037] Figure 2This is a flowchart illustrating a video processing method according to an exemplary embodiment, such as... Figure 2 As shown, the video processing method includes the following steps:

[0038] In step S201, the coding tree units of the image frames to be encoded in the target video are obtained. In this step, each image frame of the target video is first divided into coding tree units of equal size, and then encoded separately. Generally, video encoding is performed frame by frame; this disclosure uses one image frame from the target video as an example for illustration.

[0039] In step S202, for each coding tree unit, the following predetermined processing is performed: obtaining the first pixel set and the second pixel set of the current coding tree unit, wherein the first pixel set includes the pixels in the bottom four rows of the current coding tree unit before the deblocking filtering is performed, and the second pixel set is obtained after the deblocking filtering is performed on the current coding tree unit and the right-adjacent coding tree units of the current coding tree unit, and the second pixel set includes the pixels in the current coding tree unit excluding the bottom four rows after the deblocking filtering is performed; performing information statistics on the first pixel set and the second pixel set, and determining the adaptive sample compensation mode of the current coding tree unit based on the information statistics results. For example, when it is necessary to perform information statistics on the current coding tree unit, it is possible to wait until the current coding tree unit and its right-adjacent coding tree units have undergone deblocking filtering before starting to perform information statistics on the current coding tree unit. At this time, since the current coding tree unit and its right-adjacent coding tree units have all undergone deblocking filtering, the pixels in the rightmost four columns of the current coding tree unit (CTU) will not change. The rightmost four columns of deblocked pixels are used for information statistics. Since relatively accurate pixels are used, the accuracy of information statistics can be improved. That is, among the pixels used for information statistics in this disclosure, only the pixels in the bottom four rows of the current coding tree unit are the pixels before the deblocking filtering is performed, and the other pixels (including the pixels in the rightmost four columns of the current coding tree unit) are the pixels after the deblocking filtering is performed. Information statistics are performed on the pixels in the bottom four rows of the current coding tree unit before deblocking and the pixels in the remaining rows of the current coding tree unit after deblocking, to obtain the information statistics results. Based on the information statistics results, multiple adaptive sample compensation (SAO) modes can be determined. According to the rate distortion cost corresponding to multiple SAO modes, the optimal adaptive sample compensation (SAO) mode for the current coding tree unit is selected. For example, the SAO mode with the lowest rate distortion cost is selected as the optimal adaptive sample compensation (SAO) mode for the current coding tree unit.

[0040] According to an exemplary embodiment of the present disclosure, the rate-distortion cost of the current coding tree unit under at least one adaptive sample compensation (SAO) mode can be obtained based on information statistics results, wherein the at least one adaptive sample compensation (SAO) mode includes a mode without adaptive sample compensation (SAO), and the adaptive sample compensation (SAO) mode of the current coding tree unit is determined from the at least one adaptive sample compensation (SAO) mode based on the rate-distortion cost.

[0041] According to an exemplary embodiment of this disclosure, before performing the predetermined processing, the method further includes: determining whether to enable adaptive sample compensation based on relevant information of the image frame to be encoded; and performing the predetermined processing if it is determined that adaptive sample compensation is enabled. Since not all image frames in the target video undergo the predetermined processing, according to this embodiment, image frames with adaptive sample compensation enabled are determined based on relevant information of the image frames to be encoded. Only these image frames undergo the predetermined processing, thereby avoiding operations on image frames that do not require predetermined processing and reducing the amount of content to be written to the bitstream.

[0042] Specifically, Adaptive Sample Compensation (SAO) is a frame-level switch. If frame-level SAO is enabled, each CTU in the image frame needs to spend corresponding bits to record SAO-related information. If frame-level SAO is disabled, each CTU in the image frame does not need to record SAO-related information, meaning no bitrate is consumed. For image frames with many skipped pixels, the bitrate increase caused by the bit overhead of SAO-related information outweighs the benefits. Therefore, selectively disabling SAO in some frames can yield better benefits, i.e., a lower bitrate for the same quality. Thus, in this embodiment, the decision to enable adaptive sample compensation can be determined based on the relevant information of the image frame to be encoded. If adaptive sample compensation is enabled, predetermined processing is performed. Furthermore, determining whether to enable adaptive sample compensation based on the relevant information of the image frame to be encoded can be done in various ways, which will be explained below according to different relevant information. It should be noted that the relevant information given below is merely an example, and this disclosure does not limit it.

[0043] According to an exemplary embodiment of this disclosure, the relevant information includes the frame type of the image frame to be encoded and / or a preset quality coefficient, wherein the preset quality coefficient represents a pre-set coefficient used to indicate the encoding quality of the target video; determining whether to enable adaptive sample compensation based on the relevant information of the image frame to be encoded includes: determining to enable adaptive sample compensation when one of the following conditions is met: the preset quality coefficient is less than a first preset value; the preset quality coefficient is greater than or equal to the first preset value and less than a second preset value, and the frame type of the image frame to be encoded is not a non-reference B-frame; the preset quality coefficient is greater than or equal to the second preset value and the frame type of the image frame to be encoded is not a B-frame. According to this embodiment, since the number of coding tree units that do not perform adaptive sample compensation in image frames with frame type B is relatively large, enabling adaptive sample compensation for B-frames is relatively wasteful of bitrate. In this case, adaptive sample compensation can be directly disabled for image frames with frame type B, reducing bitrate waste.

[0044] To facilitate understanding, let's briefly introduce the types of image frames. Image frames include I-frames, P-frames, and B-frames. An I-frame (Intra-Picture) is a complete compressed image; it can be decompressed to obtain a complete image (largest data size). A P-frame (Predictive-Picture) is recorded when a subsequent image differs significantly from an I-frame. A P-frame records the approximate difference between itself and the preceding P-frame or I-frame. A B-frame (Bidirectional Interpolated Prediction Frame) records a motion vector based on the conditions of the next and previous frames (smallest data size). It's important to note that I-frames do not reference any other frames, while P-frames only reference the preceding frame. Any frame can use a B-frame as its reference frame. If a B-frame can be used as a reference frame, it is called a reference B-frame. If a B-frame is not used as a reference frame, it is called a non-reference B-frame.

[0045] Specifically, before encoding an image frame, it can be determined whether SAO (Self-Activated Aspect Ratio) is enabled for the image frame to be encoded, i.e., whether the aforementioned predetermined processing should be performed, based on the frame type and a preset quality factor (crf). For example, the first preset value can be set to 20 as needed, and the second preset value can be set to 26 as needed. Therefore, when crf < 20, SAO is enabled for all image frames, and the image frame to be encoded is determined to have SAO enabled. When 20 <= crf < 26, SAO is disabled for all non-reference B-frames. At this time, it is necessary to determine whether SAO is enabled based on the frame type of the image frame to be encoded. If the image frame to be encoded is a non-reference B-frame, SAO is not enabled; if the image frame to be encoded is not a non-reference B-frame, SAO is enabled. When 26 <= crf, SAO is disabled for all B-frames. At this time, it is necessary to determine whether SAO is enabled based on the frame type of the image frame to be encoded. If the image frame to be encoded is a B-frame, SAO is not enabled; if the image frame to be encoded is not a B-frame, SAO is enabled.

[0046] According to an exemplary embodiment of this disclosure, the relevant information includes the prediction mode of the coding tree units in the image frame to be encoded. Based on the relevant information of the image frame to be encoded, determining whether to enable adaptive sample compensation includes: determining the number of coding tree units in the image frame to be encoded whose prediction mode adopts the skip mode based on the identifier of the prediction mode; and determining to enable adaptive sample compensation if the number is less than a third preset value. According to this embodiment, since the probability of coding tree units whose prediction mode adopts the skip (SKIP) mode not performing adaptive sample compensation is relatively high, or they do not perform adaptive sample compensation at all, when the number of coding tree units in the image frame to be encoded using the SKIP mode exceeds a certain value, the adaptive sample compensation switch of the image frame to be encoded can be directly turned off, that is, the image frame to be encoded does not enable adaptive sample compensation, thereby reducing the waste of bitrate.

[0047] It should be noted that the above embodiments can be performed before encoding, and the above-mentioned third preset value can be set as needed, and this disclosure does not limit either. Specifically, the number of coding tree units in the image frame to be encoded that use the skip mode (i.e., SKIP mode) for prediction mode is counted. When the number of such coding tree units reaches a certain value, that is, when the above-mentioned third preset value is reached, the adaptive sample compensation (SAO) is turned off.

[0048] According to an exemplary embodiment of this disclosure, the relevant information includes the rate-distortion cost of the image frame to be encoded. Based on the relevant information of the image frame to be encoded, determining whether to enable adaptive sample compensation includes: obtaining the intra-frame rate-distortion cost of the image frame to be encoded, the inter-frame rate-distortion cost of the image frame to be encoded, and the inter-frame rate-distortion cost of the image frame to be encoded and the corresponding reference frame; and determining to enable adaptive sample compensation when the difference between the inter-frame rate-distortion cost and the intra-frame rate-distortion cost is less than a fourth preset value. According to this embodiment, since there are a relatively large number of coding tree units in image frames where the intra-frame rate-distortion cost is less than a certain value of the inter-frame rate-distortion cost, enabling adaptive sample compensation for such image frames would be wasteful of bitrate. In this case, adaptive sample compensation can be directly disabled for such image frames to reduce bitrate waste.

[0049] For example, the intra-frame rate-distortion cost (intercost) and inter-frame rate-distortion cost (intracost) can be obtained from the lookahead module, and then a decision can be made on whether to use SAO for the current image frame based on these two values. Specifically, when the intra-frame rate-distortion cost (intercost) of an image frame is less than a certain value of the inter-frame rate-distortion cost (intracost), SAO can be turned off. SAO is only turned on when the intra-frame rate-distortion cost (intercost) is not less than a certain value of the inter-frame rate-distortion cost (intracost).

[0050] return Figure 2 In step S203, the relevant information of the adaptive sample compensation mode is written into the bitstream to be sent to the decoding end. The decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode. The relevant information of the optimal adaptive sample compensation mode selected above is written into the bitstream to be sent to the decoding end so that the decoding end can perform decoding based on this relevant information. For example, based on the offset value in the relevant information, the decoding end performs sample compensation on the current coding tree unit.

[0051] According to an exemplary embodiment of this disclosure, when it is determined that adaptive sample compensation is not enabled, an identifier indicating that adaptive sample compensation is not enabled for the image frame to be encoded is written into the bitstream to be sent to the decoding end. According to this embodiment, when none of the coding tree units in the image frame have adaptive sample compensation enabled, i.e., none of them undergo the predetermined processing, it indicates that there is no corresponding compensation method. Therefore, the identifier indicating that the predetermined processing has not been performed on the image frame to be encoded can be written into the bitstream, reducing bitrate waste.

[0052] Specifically, all image frames with SAO enabled are first encoded. After encoding, the proportion of CTUs that have not performed SAO in the current image frame (i.e., the image frame to be encoded mentioned above) is counted. If none of the CTUs in the current image frame have performed SAO, the frame-level SAO flag is written to 0 in the bitstream, and the SAO information of the CTUs does not need to be written into the bitstream.

[0053] According to an exemplary embodiment of this disclosure, the number of coding tree units in the image frame to be encoded whose compensation method is not performed can also be determined based on the adaptive sample compensation mode of each coding tree unit in the determined image frame to be encoded. If the number of coding tree units in the image frame to be encoded whose compensation method is not performed exceeds a preset ratio, an identifier indicating that the image frame to be encoded has not enabled adaptive sample compensation is written into the bitstream to be sent to the decoding end. According to this embodiment, when the image frame has SAO enabled, but a certain number of coding tree units have not performed SAO, that is, a certain number of coding tree units have not performed the predetermined processing, this can also be regarded as no corresponding compensation method. Therefore, it is sufficient to write the identifier that the image frame to be encoded has not performed the predetermined processing into the bitstream to reduce bitrate waste.

[0054] Specifically, the above embodiments are generally applied in scenarios where the encoder is configured as a single thread. In this scenario, for all image frames with SAO enabled, the number of CTUs that do not perform SAO in the current image frame is detected. If the number of such CTUs accounts for more than 80% in the current image frame, the SAO of the current image frame can be turned off to avoid wasting bitrate.

[0055] Figure 3 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment. (Refer to...) Figure 3 The device includes an acquisition unit 30, a processing unit 32, and a writing unit 34.

[0056] The acquisition unit 30 is configured to acquire the coding tree units of the image frames to be encoded in the target video; the processing unit 32 is configured to perform the following predetermined processing for each coding tree unit: acquire the first pixel set and the second pixel set of the current coding tree unit, wherein the first pixel set includes the pixels in the bottom four rows of the current coding tree unit before deblocking filtering, and the second pixel set is acquired after deblocking filtering is performed on the current coding tree unit and the right-side adjacent coding tree units of the current coding tree unit, and the second pixel set includes the pixels in the current coding tree unit excluding the bottom four rows after deblocking filtering; perform information statistics on the first pixel set and the second pixel set, and determine the adaptive sample compensation mode of the current coding tree unit based on the information statistics results; the writing unit 34 is configured to write the relevant information of the adaptive sample compensation mode into the bitstream to be sent to the decoding end, wherein the decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

[0057] According to an exemplary embodiment of the present disclosure, the processing unit 32 is further configured to determine whether to enable adaptive sample compensation based on relevant information of the image frame to be encoded before performing the predetermined processing; and to perform the predetermined processing if it is determined that adaptive sample compensation is enabled.

[0058] According to an exemplary embodiment of this disclosure, the relevant information includes the frame type of the image frame to be encoded and / or a preset quality coefficient, wherein the preset quality coefficient represents a pre-set coefficient used to indicate the encoding quality of the target video; the processing unit 32 is further configured to determine to enable adaptive sample compensation if one of the following conditions is met: the preset quality coefficient is less than a first preset value; the preset quality coefficient is greater than or equal to the first preset value and less than a second preset value, and the frame type of the image frame to be encoded is not a non-reference B-frame; the preset quality coefficient is greater than or equal to the second preset value and the frame type of the image frame to be encoded is not a B-frame.

[0059] According to an exemplary embodiment of this disclosure, the relevant information includes the prediction mode of the coding tree unit in the image frame to be encoded. The processing unit 32 is further configured to determine the number of coding tree units in the image frame to be encoded whose prediction mode adopts the skip mode based on the identifier of the prediction mode in the image frame to be encoded; and if the number is less than a third preset value, determine to enable adaptive sample compensation.

[0060] According to an exemplary embodiment of this disclosure, the relevant information includes the rate-distortion cost of the image frame to be encoded. The processing unit 32 is further configured to acquire the intra-frame rate-distortion cost of the image frame to be encoded, the inter-frame rate-distortion cost of the image frame to be encoded and the corresponding reference frame; and determine to enable adaptive sampling compensation when the difference between the inter-frame rate-distortion cost and the intra-frame rate-distortion cost is less than a fourth preset value.

[0061] According to an exemplary embodiment of the present disclosure, the writing unit 34 is further configured to write an identifier indicating that adaptive sample compensation is not enabled into the bitstream to be sent to the decoding end when it is determined that adaptive sample compensation is not enabled.

[0062] According to an exemplary embodiment of this disclosure, the writing unit 34 is further configured to determine the number of coding tree units in the image frame to be encoded that are not compensated based on the adaptive sample compensation mode of each coding tree unit in the determined image frame to be encoded; and if the number of coding tree units in the image frame to be encoded that are not compensated exceeds a preset ratio, write an identifier indicating that the image frame to be encoded has not enabled adaptive sample compensation into the bitstream to be sent to the decoding end.

[0063] According to embodiments of this disclosure, an electronic device may be provided. Figure 4 This is a block diagram of an electronic device 400 according to an embodiment of the present disclosure. The electronic device includes at least one memory 401 and at least one processor 402. The at least one memory stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor, a video processing method according to an embodiment of the present disclosure is performed.

[0064] As an example, electronic device 400 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 1000 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 400 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0065] In electronic device 400, processor 402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 402 may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0066] The processor 402 can execute instructions or code stored in memory, wherein memory 401 can also store data. Instructions and data can also be sent and received over a network via a network interface device, wherein the network interface device can employ any known transmission protocol.

[0067] The memory 401 may be integrated with the processor 402, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 401 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 401 and the processor 402 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 402 to read files stored in the memory 401.

[0068] In addition, the electronic device 400 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device can be interconnected via a bus and / or network.

[0069] According to embodiments of this disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the video processing method of the embodiments of this disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0070] According to an embodiment of this disclosure, a computer program product is provided, including computer instructions, which, when executed by a processor, implement the video processing method of the embodiment of this disclosure.

[0071] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0072] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, include: Obtain the coding tree units of the image frames to be encoded in the target video; For each coding tree unit, the following pre-defined processing is performed: Obtain the first pixel set and the second pixel set of the current coding tree unit, wherein the first pixel set includes the pixels in the bottom four rows of the current coding tree unit before performing deblocking filtering, and the second pixel set is obtained after performing deblocking filtering on the current coding tree unit and the right adjacent coding tree unit of the current coding tree unit, and the second pixel set includes the pixels in the current coding tree unit excluding the bottom four rows after performing deblocking filtering; Information statistics are performed on the first pixel set and the second pixel set, and the adaptive sample compensation mode of the current coding tree unit is determined based on the information statistics results; The relevant information of the adaptive sample compensation mode is written into the bitstream to be sent to the decoding end, wherein the decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

2. The video processing method as described in claim 1, characterized in that, Before performing the predetermined process, the following is also included: Based on the relevant information of the image frame to be encoded, determine whether to enable adaptive sample compensation; If adaptive sample point compensation is enabled, the predetermined process is executed.

3. The video processing method as described in claim 2, characterized in that, The relevant information includes the frame type of the image frame to be encoded and / or a preset quality coefficient, wherein the preset quality coefficient represents a pre-set coefficient used to indicate the encoding quality of the target video; The step of determining whether to enable adaptive sample compensation based on the relevant information of the image frame to be encoded includes: Adaptive sample compensation is enabled if one of the following conditions is met: The preset quality coefficient is less than the first preset value; The preset quality coefficient is greater than or equal to the first preset value and less than the second preset value, and the frame type of the image frame to be encoded is not a non-reference B-frame; The preset quality coefficient is greater than or equal to the second preset value and the frame type of the image frame to be encoded is not a B-frame.

4. The video processing method as described in claim 2, characterized in that, The relevant information includes the prediction mode of the coding tree unit in the image frame to be encoded. The step of determining whether to enable adaptive sample compensation based on the relevant information of the image frame to be encoded includes: Based on the identifier of the prediction mode in the image frame to be encoded, determine the number of coding tree units in the image frame to be encoded whose prediction mode adopts the skip mode; If the number is less than a third preset value, then adaptive sample point compensation is enabled.

5. The video processing method as described in claim 2, characterized in that, The relevant information includes the rate-distortion cost of the image frame to be encoded. The step of determining whether to enable adaptive sample compensation based on the relevant information of the image frame to be encoded includes: Obtain the intra-frame rate-distortion cost of the image frame to be encoded, and the inter-frame rate-distortion cost of the image frame to be encoded and the corresponding reference frame; If the difference between the inter-frame rate distortion cost and the intra-frame rate distortion cost is less than a fourth preset value, then adaptive sample compensation is enabled.

6. The video processing method according to any one of claims 2 to 5, characterized in that, Also includes: If it is determined that adaptive sample compensation is not enabled, an identifier indicating that the image frame to be encoded is not enabled is written into the bitstream to be sent to the decoding end.

7. The video processing method as described in claim 1, characterized in that, The method further includes: Based on the determined adaptive sample compensation mode of each coding tree unit in the image frame to be encoded, the number of coding tree units in the image frame to be encoded whose compensation method is no compensation is determined. If the number of coding tree units in the image frame to be encoded that are not compensated exceeds a preset ratio, an identifier indicating that adaptive sample compensation is not enabled in the image frame to be encoded will be written into the bitstream to be sent to the decoding end.

8. A video processing apparatus, characterized in that, include: The acquisition unit is a coding tree unit configured to acquire the image frames to be encoded in the target video; The processing unit is configured to perform the following predetermined processing for each coding tree unit: Obtain the first pixel set and the second pixel set of the current coding tree unit. The first pixel set includes the pixels in the bottom four rows of the current coding tree unit before deblocking filtering is performed. The second pixel set is obtained after deblocking filtering is performed on the current coding tree unit and the right-adjacent coding tree units of the current coding tree unit. The second pixel set includes the pixels in the current coding tree unit after deblocking filtering, excluding the bottom four rows. Perform information statistics on the first pixel set and the second pixel set. Based on the information statistics results, determine the adaptive sample compensation mode of the current coding tree unit. The writing unit is configured to write relevant information of the adaptive sample compensation mode into the bitstream to be sent to the decoding end, wherein the decoding end is used to perform sample compensation on the current coding tree unit based on the adaptive sample compensation mode.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the at least one processor to perform the video processing method as described in any one of claims 1 to 7.

Images