Video frame encoding methods, devices, electronic equipment and storage media

Abstract

This disclosure provides a video frame encoding method, apparatus, electronic device, and storage medium, belonging to the field of computer technology. The method includes: for any video frame in a video frame queue, filtering the video frame based on multiple reference video frames to obtain a filtered video frame and a filtering quantization loss; pre-coding the video frame and the filtered video frame based on the multiple reference video frames to determine multiple first quantization losses and multiple second quantization losses; encoding the video frame if the sum of the multiple reference losses is less than the filtering quantization loss, where the reference loss is the difference between the corresponding first and second quantization losses. This scheme, by comparing the sum of the differences between the multiple quantization losses generated during pre-coding of the video frames before and after filtering with the magnitude of the filtering quantization loss, can adaptively determine whether to retain the filtered video frame, thereby improving the video encoding quality.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a video frame encoding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Video consists of consecutive video frames, and these frames share similarities. Therefore, to facilitate video storage and transmission, video needs to be encoded to reduce its storage space. Improving video encoding efficiency is a technical problem that needs to be solved.

[0003] In video coding techniques, Motion Compensated Temporal Filtering (MCTF) utilizes the temporal correlation of video to perform temporal filtering on video frames, thereby reducing temporal redundancy information generated during video frame referencing and improving video coding efficiency.

[0004] However, this scheme determines whether to perform temporal filtering on video frames solely based on the magnitude of the quantization parameters, ignoring the diversity of videos. For video frames in different video types, filtering is performed when the quantization parameters of the video frame exceed a certain value, which can lead to problems such as color distortion and blockiness in the filtered video frames, thereby reducing the encoded quality of the video. Summary of the Invention

[0005] This disclosure provides a video frame encoding method, apparatus, electronic device, and storage medium. By comparing the sum of differences between multiple quantization losses generated during pre-coding of video frames before and after filtering with the magnitude of the filtering quantization loss, it can adaptively determine whether to retain the filtered video frame of the current video frame, thereby improving the video encoding quality. The technical solution of this disclosure is as follows:

[0006] According to one aspect of the embodiments of this disclosure, a video frame encoding method is provided, including:

[0007] For any video frame in the video frame queue, the video frame is filtered based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame.

[0008] Based on the plurality of reference video frames, the video frame and the filtered video frame are pre-coded respectively to determine a plurality of first quantization losses and a plurality of second quantization losses. The plurality of first quantization losses are used to indicate the quantization error between the video frame and the plurality of reference video frames respectively, and the plurality of second quantization losses are used to indicate the quantization error between the second video frame and the plurality of reference video frames respectively. The plurality of first quantization losses and the plurality of second quantization losses correspond one-to-one.

[0009] The video frame is encoded when the sum of multiple reference losses is less than the filtering quantization loss, wherein the reference loss is the difference between the corresponding first quantization loss and second quantization loss.

[0010] According to another aspect of the embodiments of this disclosure, a video frame encoding apparatus is provided, comprising:

[0011] The filtering unit is configured to filter any video frame in the video frame queue based on multiple reference video frames to obtain a filtered video frame and a filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame.

[0012] The precoding unit is configured to precode the video frame and the filtered video frame based on the plurality of reference video frames to determine a plurality of first quantization losses and a plurality of second quantization losses. The plurality of first quantization losses are used to indicate the quantization error between the video frame and the plurality of reference video frames, and the plurality of second quantization losses are used to indicate the quantization error between the second video frame and the plurality of reference video frames, respectively. The plurality of first quantization losses and the plurality of second quantization losses correspond one-to-one.

[0013] The encoding unit is configured to encode the video frame when the sum of a plurality of reference losses is less than the filter quantization loss, wherein the reference loss is the difference between the corresponding first quantization loss and second quantization loss.

[0014] In some embodiments, the filtering unit includes:

[0015] The filtering subunit is configured to filter any video frame in the video frame queue based on multiple reference video frames to obtain the filtered video frame.

[0016] The first segmentation subunit is configured to segment the video frame into multiple first pixel blocks;

[0017] The second segmentation subunit is configured to segment the filtered video frame into multiple second pixel blocks, and the multiple second pixel blocks correspond one-to-one with the multiple first pixel blocks.

[0018] The first determining subunit is configured to determine the filtering quantization loss based on the plurality of first pixel blocks and the plurality of second pixel blocks.

[0019] In some embodiments, the first determining subunit is configured to, for any first pixel block, subtract the first pixel block from the corresponding second pixel block to obtain a residual block of the first pixel block; pre-encode the multiple residual blocks of the multiple first pixel blocks; for any pre-encoded residual block, obtain the mean square sum of the residual block based on the multiple pixel values ​​of the residual block; and determine the average value of the mean square sum of the multiple residual blocks as the filtering quantization loss.

[0020] In some embodiments, the precoding unit includes:

[0021] The third segmentation subunit is configured to segment any reference video frame into multiple reference pixel blocks, and the multiple reference pixel blocks correspond one-to-one with the multiple first pixel blocks.

[0022] The second determining subunit is configured to determine a first quantization loss between the reference video frame and the video frame based on the plurality of reference pixel blocks and the plurality of first pixel blocks;

[0023] The third determining subunit is configured to determine a second quantization loss between the reference video frame and the filtered video frame based on the plurality of reference pixel blocks and the plurality of second pixel blocks.

[0024] In some embodiments, the second determining subunit is configured to, for any reference pixel block, subtract the reference pixel block from the corresponding first pixel block to obtain a first residual block of the reference pixel block; pre-encode the plurality of first residual blocks of the plurality of reference pixel blocks; for any pre-encoded first residual block, determine the mean square sum of the first residual block based on the plurality of pixel values ​​of the first residual block; and determine the average value of the mean square sum of the plurality of first residual blocks as the first quantization loss.

[0025] In some embodiments, the third determining subunit is configured to, for any reference pixel block, subtract the reference pixel block from the corresponding second pixel block to obtain a second residual block of the reference pixel block; pre-encode the multiple second residual blocks of the multiple reference pixel blocks; for any pre-encoded second residual block, determine the mean square sum of the second residual block based on the multiple pixel values ​​of the second residual block; and determine the average value of the mean square sum of the multiple second residual blocks as the second quantization loss.

[0026] In some embodiments, the encoding unit is further configured to encode the filtered video frame if the sum of the plurality of reference losses is not less than the filtering quantization loss.

[0027] According to another aspect of the embodiments of this disclosure, an electronic device is provided, the electronic device comprising:

[0028] One or more processors;

[0029] Memory used to store the executable program code of the processor;

[0030] The processor is configured to execute the program code to implement the aforementioned video frame encoding method.

[0031] According to another aspect of the present disclosure, a computer-readable storage medium is provided that, when program code in the computer-readable storage medium is executed by a processor of an electronic device, enables the electronic device to perform the video frame encoding method described above.

[0032] According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described video frame encoding method.

[0033] This disclosure provides a video frame encoding method. By filtering a video frame, a filtered video frame and a filtering quantization loss can be obtained. Based on multiple reference video frames, the video frame and the filtered video frame are pre-encoded separately. Multiple first quantization losses generated during the video frame pre-encoding process and multiple second quantization losses generated during the filtered video frame pre-encoding process can be determined based on the pre-encoded video frame and the filtered video frame. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the loss generated during video frame pre-encoding is less than the total loss generated during filtering and pre-encoding of the filtered video frame. Therefore, it can be inferred that the loss generated during the formal encoding of the video frame is less than the total loss generated during filtering and pre-encoding of the filtered video frame. In this case, the electronic device can achieve a smaller loss during the formal encoding process, thus improving the video encoding quality. By comparing the sum of the differences between the multiple quantization losses generated during the pre-encoding process of the video frames before and after filtering with the magnitude of the filtering quantization loss, the electronic device can adaptively determine whether to retain the filtered video frame of the current video frame, thereby improving the video encoding quality.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0036] Figure 1 This is a schematic diagram illustrating the implementation environment of a video frame encoding method according to an exemplary embodiment.

[0037] Figure 2 This is a flowchart illustrating a video frame encoding method according to an exemplary embodiment.

[0038] Figure 3 This is a flowchart illustrating another video frame encoding method according to an exemplary embodiment.

[0039] Figure 4 This is a block diagram illustrating a video frame encoding apparatus according to an exemplary embodiment.

[0040] Figure 5 This is a block diagram illustrating another video frame encoding apparatus according to an exemplary embodiment.

[0041] Figure 6 This is a block diagram illustrating a terminal according to an exemplary embodiment.

[0042] Figure 7 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0043] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0044] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0045] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the video frames, reference video frames, and filtered video frames involved in this disclosure were all obtained with full authorization.

[0046] Figure 1 This is a schematic diagram illustrating an implementation environment for a video frame encoding method according to an exemplary embodiment. See also... Figure 1 The implementation environment specifically includes: terminal 101 and server 102.

[0047] Terminal 101 can be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), and laptop computer. An application can be installed and run on terminal 101, and users can log in to the application through terminal 101 to access the services provided by the application. Terminal 101 can be connected to server 102 via a wireless network or a wired network.

[0048] Terminal 101 can refer to one of a plurality of terminals, and this embodiment uses terminal 101 as an example. Those skilled in the art will know that the number of terminals can be more or less. For example, there can be several terminals, or dozens or hundreds of terminals, or more. This embodiment does not limit the number of terminals or the type of devices.

[0049] Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, server 102 undertakes the main computing work, and terminal 101 undertakes the secondary computing work; or, server 102 undertakes the secondary computing work, and terminal 101 undertakes the main computing work; or, server 102 and terminal 101 collaborate on computing using a distributed computing architecture. Server 102 can connect to terminal 101 and other terminals via a wireless network or wired network. Optionally, the number of servers can be more or less, and this disclosure does not limit this. Of course, server 102 may also include other functional servers to provide more comprehensive and diversified services.

[0050] Figure 2 This is a flowchart illustrating a video frame encoding method according to an exemplary embodiment, such as... Figure 2 As shown, the method is performed by an electronic device and includes the following steps:

[0051] In step S201, for any video frame in the video frame queue, the electronic device filters the video frame based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame.

[0052] In this embodiment, the video frame queue can be a segment of the video to be encoded or a complete video. The video frame queue includes multiple consecutive video frames. For any video frame in the video frame queue, the electronic device can determine multiple reference video frames from the video frame queue. Since the multiple reference video frames are used to provide reference information during the filtering process of the video frame, the electronic device can filter the video frame based on the multiple reference video frames to obtain the filtered video frame, and then determine the filtering quantization loss generated during the filtering process based on the difference between the video frame and the filtered video frame. Optionally, the multiple reference video frames are the neighboring video frames of the video frame, such as two video frames before and after the video frame; the number of multiple reference video frames can be 4, 6, 8, etc., and this embodiment does not limit this.

[0053] For example, for the 8th video frame in the video frame queue, if the number of multiple reference video frames is 8, then the multiple reference video frames for this video frame are the 4th, 5th, 6th, and 7th video frames and the 9th, 10th, 11th, and 12th video frames in the video frame queue.

[0054] In step S202, the electronic device pre-encodes the video frame and the filtered video frame based on multiple reference video frames to determine multiple first quantization losses and multiple second quantization losses. The multiple first quantization losses are used to indicate the quantization error between the video frame and the multiple reference video frames, respectively, and the multiple second quantization losses are used to indicate the quantization error between the second video frame and the multiple reference video frames, respectively. The multiple first quantization losses and the multiple second quantization losses correspond one-to-one.

[0055] In this embodiment of the disclosure, when encoding any video frame in the video frame queue, the electronic device needs to utilize the temporal correlation between the video frame and multiple reference video frames to encode the video frame, thereby achieving video compression. Therefore, before formally encoding the video frame or filtered video frame, the electronic device can pre-encode the video frame and the filtered video frame based on multiple reference video frames, respectively, to obtain pre-encoded video frames and pre-encoded filtered video frames. Based on the differences between the video frame and the pre-encoded video frames, the electronic device can determine multiple first quantization losses generated during the pre-encoding process of the video frame. Similarly, based on the differences between the filtered video frame and the pre-encoded filtered video frame, the electronic device can determine multiple second quantization losses generated during the pre-encoding process of the filtered video frame. Pre-encoding includes discrete cosine transform, quantization, inverse quantization, and inverse discrete cosine transform, and is a part of the formal encoding process.

[0056] In step S203, when the sum of multiple reference losses is less than the filtering quantization loss, the electronic device encodes the video frame, and the reference loss is the difference between the corresponding first quantization loss and second quantization loss.

[0057] In this embodiment, the electronic device pre-encodes video frames and filtered video frames separately, enabling it to estimate the losses incurred during the formal encoding of both. Multiple first quantization losses are estimates of the losses incurred during the formal encoding of video frames, and multiple second quantization losses are estimates of the losses incurred during the formal encoding of filtered video frames. Based on the correspondence between the first and second quantization losses, the electronic device determines multiple reference losses by determining the differences between them. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the losses incurred during video frame pre-encoding are less than the total losses incurred during the pre-encoding and filtering of video frames. Therefore, it can be inferred that the losses incurred during formal video frame encoding are less than the total losses incurred during the formal encoding and filtering of video frames, resulting in lower losses during video frame encoding and improved encoding quality compared to encoding filtered video frames.

[0058] This disclosure provides a video frame encoding method. By filtering a video frame, a filtered video frame and a filtering quantization loss can be obtained. Based on multiple reference video frames, the video frame and the filtered video frame are pre-encoded separately. Multiple first quantization losses generated during the video frame pre-encoding process and multiple second quantization losses generated during the filtered video frame pre-encoding process can be determined based on the pre-encoded video frame and the filtered video frame. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the loss generated during video frame pre-encoding is less than the total loss generated during filtering and pre-encoding of the filtered video frame. Therefore, it can be inferred that the loss generated during the formal encoding of the video frame is less than the total loss generated during filtering and pre-encoding of the filtered video frame. In this case, the electronic device can achieve a smaller loss during the formal encoding process, thus improving the video encoding quality. By comparing the sum of the differences between the multiple quantization losses generated during the pre-encoding process of the video frames before and after filtering with the magnitude of the filtering quantization loss, the electronic device can adaptively determine whether to retain the filtered video frame of the current video frame, thereby improving the video encoding quality.

[0059] In some embodiments, for any video frame in the video frame queue, the video frame is filtered based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss, including:

[0060] For any video frame in the video frame queue, the video frame is filtered based on multiple reference video frames to obtain a filtered video frame.

[0061] The video frame is divided into blocks to obtain multiple first pixel blocks;

[0062] The filtered video frame is divided into blocks to obtain multiple second pixel blocks, and each of the multiple second pixel blocks corresponds one-to-one with a multiple first pixel block.

[0063] The filtering quantization loss is determined based on multiple first pixel blocks and multiple second pixel blocks.

[0064] In this embodiment of the disclosure, the electronic device, based on the filtering quantization loss, can determine whether filtering video frames can bring positive benefits to the encoding process, thereby achieving higher video encoding quality, by considering the relationship between the filtering quantization loss and the sum of the differences between the filtering quantization loss and the sum of the differences between the multiple first quantization losses generated by precoding the video frame and the multiple second quantization losses generated by precoding the filtered video frame.

[0065] In some embodiments, determining the filtering quantization loss based on a plurality of first pixel blocks and a plurality of second pixel blocks includes:

[0066] For any first pixel block, subtract the corresponding second pixel block by pixels to obtain the residual block of the first pixel block;

[0067] Pre-encode multiple residual blocks of multiple first pixel blocks;

[0068] For any residual block after precoding, the mean square sum of the residual block is obtained based on multiple pixel values ​​of the residual block;

[0069] The average value of the sum of the mean squares of multiple residual blocks is determined as the filtering quantization loss.

[0070] In this embodiment of the disclosure, the electronic device can determine the average value of the mean square sum of multiple residual blocks as the quantization error between the video frame and the filtered video frame, i.e., the filtering quantization loss. Based on the filtering quantization loss, the electronic device can determine whether filtering the video frame can bring positive benefits to the encoding process, thereby achieving higher video encoding quality, by considering the relationship between the filtering quantization loss and the sum of the differences between multiple first quantization losses generated by precoding the video frame and multiple second quantization losses generated by precoding the filtered video frame.

[0071] In some embodiments, based on multiple reference video frames, precoding is performed on video frames and filtered video frames respectively to determine multiple first quantization losses and multiple second quantization losses, including:

[0072] For any reference video frame, the reference video frame is divided into blocks to obtain multiple reference pixel blocks, and the multiple reference pixel blocks correspond one-to-one with multiple first pixel blocks;

[0073] Based on multiple reference pixel blocks and multiple first pixel blocks, the first quantization loss between reference video frames and video frames is determined;

[0074] A second quantization loss is determined between a reference video frame and a filtered video frame based on multiple reference pixel blocks and multiple second pixel blocks.

[0075] In this embodiment of the disclosure, the electronic device is able to estimate the loss generated when formally encoding a video frame and the loss generated when formally encoding a filtered video frame, based on a plurality of first quantization losses and a plurality of second quantization losses. Then, by comparing the sum of the differences between the plurality of first quantization losses and the plurality of second quantization losses with the magnitude of the filtered quantization loss, it can determine whether encoding a video frame or encoding a filtered video frame results in less loss, thereby achieving higher video encoding quality.

[0076] In some embodiments, determining a first quantization loss between reference video frames and video frames based on a plurality of reference pixel blocks and a plurality of first pixel blocks includes:

[0077] For any reference pixel block, subtract the reference pixel block from the corresponding first pixel block to obtain the first residual block of the reference pixel block;

[0078] Pre-encode multiple first residual blocks of multiple reference pixel blocks;

[0079] For any first residual block after precoding, the mean square sum of the first residual block is determined based on multiple pixel values ​​of the first residual block;

[0080] The average of the sum of the mean squares of multiple first residual blocks is determined as the first quantization loss.

[0081] In this embodiment of the disclosure, the electronic device can determine the average value of the mean square sum of multiple first residual blocks as the quantization error between the reference video frame and the video frame, i.e., the first quantization loss. Based on the multiple first quantization losses, the electronic device can estimate the loss generated during the formal encoding of the video frame, and then determine whether to retain the filtered video frame by comparing the sum of the differences between the multiple first quantization losses and the multiple second quantization losses generated by precoding the filtered video frame with the magnitude of the filtered quantization loss, thereby achieving higher video encoding quality.

[0082] In some embodiments, a second quantization loss is determined between a reference video frame and a filtered video frame based on a plurality of reference pixel blocks and a plurality of second pixel blocks, including:

[0083] For any reference pixel block, subtract the reference pixel block from the corresponding second pixel block to obtain the second residual block of the reference pixel block;

[0084] Pre-encode multiple second residual blocks of multiple reference pixel blocks;

[0085] For any second residual block after precoding, the mean square sum of the second residual block is determined based on multiple pixel values ​​of the second residual block;

[0086] The average of the sum of the mean squares of multiple second residual blocks is determined as the second quantization loss.

[0087] In this embodiment of the disclosure, the electronic device can determine the average value of the sum of the mean squares of multiple second residual blocks as the quantization error between the filtered video frame and the reference video frame, i.e., the second quantization loss. Based on the multiple second quantization losses, the electronic device can estimate the loss generated during the formal encoding of the filtered video frame, and then determine whether to retain the filtered video frame by comparing the sum of the differences between the multiple first quantization losses and the multiple second quantization losses with the magnitude of the filtered quantization loss, thereby achieving higher video encoding quality.

[0088] In some embodiments, the method further includes:

[0089] The filtered video frame is encoded when the sum of multiple reference losses is not less than the filtering quantization loss.

[0090] In this embodiment of the disclosure, when the sum of multiple reference losses is not less than the filtering quantization loss, it indicates that the loss generated during video frame precoding is not less than the total loss generated during precoding of the filtered video frame and during video frame filtering. Therefore, it can be inferred that the loss generated during formal encoding of the video frame is not less than the total loss generated during formal encoding of the filtered video frame and during video frame filtering. In this case, the loss generated when the electronic device encodes the filtered video frame is smaller, and the encoding quality of the video frame can be improved compared to encoding the video frame directly.

[0091] The above Figure 2 The diagram illustrates the basic process of this disclosure. The following section, based on one implementation method, further elaborates on the solution provided in this disclosure. Figure 3 This is a flowchart illustrating another video frame encoding method according to an exemplary embodiment. The method is performed by an electronic device; see [link to relevant documentation]. Figure 3 The method includes:

[0092] In step S301, for any video frame in the video frame queue, the electronic device filters the video frame based on multiple reference video frames to obtain a filtered video frame. The reference video frames are used to provide reference information during the filtering process of the video frame.

[0093] In this embodiment, the video frame queue can be a segment of the video to be encoded or a complete video. The video frame queue includes multiple consecutive video frames. For any video frame in the video frame queue, the electronic device can determine multiple reference video frames from the video frame queue. These multiple reference video frames provide reference information during the filtering process of the video frame. Based on the multiple reference video frames, the electronic device can filter the video frame to obtain a filtered video frame. Optionally, the multiple reference video frames are adjacent video frames of the video frame, such as two video frames before and after the video frame; the number of multiple reference video frames can be 4, 6, 8, etc., and this embodiment does not limit this.

[0094] In step S302, the electronic device divides the video frame into blocks to obtain multiple first pixel blocks.

[0095] In this embodiment of the disclosure, the electronic device divides the video frame into blocks, using pixels as the unit, to obtain multiple first pixel blocks. Each first pixel block is of the same size.

[0096] For example, if the size of a video frame is 16x16, and the electronic device divides the video frame into 8x8 pixel units, it can obtain four first pixel blocks of size 8x8.

[0097] In step S303, the electronic device divides the filtered video frame into blocks to obtain multiple second pixel blocks, and the multiple second pixel blocks correspond one-to-one with the multiple first pixel blocks.

[0098] In this embodiment of the disclosure, similar to step S302 described above, the electronic device divides the filtered video frame into blocks, using pixels as the unit, to obtain multiple second pixel blocks. For any first pixel block, the electronic device can determine a second pixel block corresponding to the first pixel block from a filtered video frame of the same size as the video frame. The position of the second pixel block in the filtered video frame is the same as the position of the first pixel block in the video frame. Each second pixel block has the same size, and the size of the second pixel block is also the same as that of the first pixel block.

[0099] In step S304, the electronic device determines the filtering quantization loss based on multiple first pixel blocks and multiple second pixel blocks. The filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame.

[0100] In this embodiment, the video frame includes multiple first pixel blocks, and the filtered video frame includes multiple second pixel blocks, with a one-to-one correspondence between the multiple first pixel blocks and the multiple second pixel blocks. Based on the multiple first pixel blocks and the multiple second pixel blocks, the electronic device can determine multiple residual blocks. By processing these residual blocks, the quantization error between the video frame and the filtered video frame, i.e., the filtering quantization loss, can be determined. Based on the filtering quantization loss, the electronic device can determine whether filtering the video frame brings positive benefits to the encoding process, thereby achieving higher video encoding quality, by considering the relationship between the filtering quantization loss and the sum of the differences between the multiple first quantization losses generated by pre-encoding the video frame and the multiple second quantization losses generated by pre-encoding the filtered video frame.

[0101] In some embodiments, the electronic device can determine the quantization error between a video frame and a filtered video frame by pre-encoding the residual block obtained by subtracting the first pixel block from the corresponding second pixel block. Accordingly, for any first pixel block, the electronic device subtracts the first pixel block from the corresponding second pixel block to obtain a residual block of the first pixel block; then, it pre-encodes multiple residual blocks of multiple first pixel blocks; for any pre-encoded residual block, the electronic device obtains the mean square sum of the residual block based on the multiple pixel values ​​of the residual block; and the average value of the mean square sum of the multiple residual blocks is determined as the filtering quantization loss. Here, the first pixel block includes multiple pixels of the video frame, the second pixel block includes multiple pixels of the filtered video frame, and the multiple pixels of the first pixel block correspond one-to-one with the multiple pixels of the second pixel block. For any first pixel block, the electronic device obtains a residual block of the first pixel block by subtracting the values ​​of multiple pixels in the first pixel block from the values ​​of multiple pixels in the corresponding second pixel block pixel by pixel. This residual block can be used to represent the difference between the first pixel block and the second pixel block. The pixel-by-pixel subtraction is performed by subtracting the values ​​of multiple pixels in the corresponding second pixel block from the values ​​of multiple pixels in the first pixel block. By pre-encoding multiple residual blocks of multiple first pixel blocks, the electronic device can determine the mean square sum of the residual blocks based on the values ​​of multiple pixels in any pre-encoded residual block by squaring and summing the values ​​of each pixel. This mean square sum of the residual blocks is used to represent the quantization error between the first pixel block and the second pixel block. In this embodiment of the present disclosure, the electronic device can determine the average value of the mean square sum of the multiple residual blocks as the quantization error between the video frame and the filtered video frame, i.e., the filtering quantization loss. Electronic devices, based on filtering quantization loss, can determine whether filtering video frames brings positive benefits to the encoding process by considering the sum of the differences between the filtering quantization loss and the sum of the differences between multiple first quantization losses generated by precoding video frames and multiple second quantization losses generated by precoding filtered video frames, thereby achieving higher video encoding quality.

[0102] For example, both the first and second pixel blocks are 8x8 pixels in size, and each block contains 64 pixels. The electronic device subtracts the first pixel block from its corresponding second pixel block to obtain a residual block, which is also 8x8 pixels in size and contains 64 pixels. The electronic device then squares and sums the values ​​of the 64 pixels in the pre-encoded residual block to obtain the mean square sum of the residual block.

[0103] Optionally, the electronic device can determine the quantization error between the video frame and the filtered video frame, i.e. the filtering quantization loss, based on the mean square sum of multiple residual blocks according to the following formula (1).

[0104]

[0105] Among them, quantFrameSSE(F n ,F′ n ) represents the current video frame F n With the filtered video frame F′ n Quantization error between; quantSSE k For residual block ΔB′ k The mean square sum, ΔB k For the k-th residual block, ΔB′ k is the k-th residual block after precoding; m is the number of residual blocks.

[0106] In step S305, for any reference video frame, the electronic device divides the reference video frame into blocks to obtain multiple reference pixel blocks, and the multiple reference pixel blocks correspond one-to-one with multiple first pixel blocks.

[0107] In this embodiment of the disclosure, similar to step S302 described above, the electronic device divides the reference video frame into blocks, using pixels as the unit, to obtain multiple reference pixel blocks. For any first pixel block, the electronic device can determine the reference pixel block corresponding to the first pixel block from a reference video frame of the same size as the video frame. Each reference pixel block has the same size, and the size of each reference pixel block is also the same as that of the first pixel block.

[0108] In step S306, the electronic device determines a first quantization loss between a reference video frame and a video frame based on a plurality of reference pixel blocks and a plurality of first pixel blocks. The first quantization loss is used to indicate the quantization error between the video frame and the reference video frame.

[0109] In this embodiment, the video frame includes a plurality of first pixel blocks, and the reference video frame includes a plurality of reference pixel blocks, with each of the first pixel blocks corresponding one-to-one with the reference pixel blocks. Based on the plurality of first pixel blocks and the plurality of reference pixel blocks, the electronic device can determine a plurality of residual blocks. By pre-coding the plurality of residual blocks, it can determine the quantization error between the video frame and the reference video frame, i.e., the first quantization loss, based on the pre-coded residual blocks. The plurality of first quantization losses are estimates of the loss generated during the formal encoding of the video frame.

[0110] In some embodiments, the electronic device can determine the quantization error between a video frame and a reference video frame by pre-coding a first residual block obtained by subtracting a first pixel block from its corresponding reference pixel block. Accordingly, for any reference pixel block, the reference pixel block and its corresponding first pixel block are subtracted pixel by pixel to obtain a first residual block of the reference pixel block; multiple first residual blocks of multiple reference pixel blocks are pre-coded; for any pre-coded first residual block, the mean square sum of the first residual block is determined based on the multiple pixel values ​​of the first residual block; the average value of the mean square sum of the multiple first residual blocks is determined as the first quantization loss. Here, the first pixel block includes multiple pixels of the video frame, the reference pixel block includes multiple pixels of the reference video frame, and the multiple pixels of the first pixel block correspond one-to-one with the multiple pixels of the reference pixel block. For any reference pixel block, the electronic device obtains a first residual block of the reference pixel block by subtracting the values ​​of multiple pixels of the reference pixel block from the values ​​of multiple pixels of the corresponding first pixel block pixel by pixel. This first residual block represents the difference between the reference pixel block and the first pixel block. The pixel-by-pixel subtraction involves subtracting the values ​​of multiple pixels of the corresponding first pixel block from the values ​​of multiple pixels of the reference pixel block. By pre-encoding multiple first residual blocks of multiple reference pixel blocks, the electronic device can determine the mean square sum of the first residual blocks based on the squared values ​​of multiple pixels in any pre-encoded first residual block, and summing these values. This mean square sum of the first residual blocks represents the quantization error between the reference pixel block and the first pixel block. In this embodiment, the electronic device can determine the average value of the mean square sums of the multiple first residual blocks as the quantization error between reference video frames and video frames, i.e., the first quantization loss. Electronic devices can estimate the loss generated during the formal encoding of video frames based on multiple first quantization losses. They can then determine whether to retain a filtered video frame by comparing the sum of the differences between the multiple first quantization losses and the multiple second quantization losses generated during the precoding of the filtered video frame with the magnitude of the filtered quantization loss, thereby achieving higher video coding quality.

[0111] For example, both the first pixel block and the reference pixel block are 8x8 pixels in size, and each block contains 64 pixels. The electronic device subtracts the corresponding first pixel block from the reference pixel block to obtain the first residual block, which is also 8x8 pixels in size and contains 64 pixels. The electronic device can obtain the mean square sum of the 64 pixels in the pre-encoded first residual block by squaring and summing the values.

[0112] Optionally, the electronic device can determine the quantization error, i.e. the first quantization loss, between the current video frame and the reference video frame based on the mean square sum of multiple first residual blocks according to the following formula (2).

[0113]

[0114] Among them, quantFrameSSE(F n ,F i ) represents the current video frame F n With reference video frame F i The quantization error between the reference video frame F i When the quantity is 8, i = n-4, n-3, n-2, n-1, n+1, n+2, n+3, n+4; quantSSE k For the first residual block ΔB′ k The mean square sum, ΔB k For the k-th first residual block, ΔB′ k is the k-th first residual block after precoding; m is the number of first residual blocks.

[0115] In step S307, the electronic device determines a second quantization loss between the reference video frame and the filtered video frame based on multiple reference pixel blocks and multiple second pixel blocks. The second quantization loss is used to indicate the quantization error between the second video frame and the reference video frame.

[0116] In this embodiment, the filtered video frame includes multiple second pixel blocks, and the reference video frame includes multiple reference pixel blocks. Since the multiple second pixel blocks correspond one-to-one with the multiple first pixel blocks, and the multiple first pixel blocks correspond one-to-one with the multiple reference pixel blocks, the multiple second pixel blocks also correspond one-to-one with the multiple reference pixel blocks. Based on the multiple second pixel blocks and the multiple reference pixel blocks, the electronic device can determine multiple residual blocks. By pre-coding the multiple residual blocks, it can determine the quantization error between the filtered video frame and the reference video frame, i.e., the second quantization loss, based on the pre-coded residual blocks. The multiple second quantization losses are estimates of the loss generated when the filtered video frame is formally encoded. Based on the multiple first quantization losses and the multiple second quantization losses, the electronic device can estimate the loss generated when formally encoding the video frame and the loss generated when formally encoding the filtered video frame, respectively. Furthermore, by comparing the sum of the differences between the multiple first quantization losses and the multiple second quantization losses with the magnitude of the filtered quantization loss, it can determine whether encoding the video frame or encoding the filtered video frame results in less loss, thereby achieving higher video encoding quality.

[0117] In some embodiments, the electronic device can determine the quantization error between the filtered video frame and the reference video frame by pre-coding a second residual block obtained by subtracting the second pixel block from the corresponding reference pixel block. Accordingly, for any reference pixel block, the reference pixel block and the corresponding second pixel block are subtracted pixel by pixel to obtain a second residual block of the reference pixel block; multiple second residual blocks of multiple reference pixel blocks are pre-coded; for any pre-coded second residual block, the mean square sum of the second residual block is determined based on the multiple pixel values ​​of the second residual block; the average value of the mean square sum of the multiple second residual blocks is determined as the second quantization loss. Here, the second pixel block includes multiple pixels of the filtered video frame, the reference pixel block includes multiple pixels of the reference video frame, and the multiple pixels of the second pixel block correspond one-to-one with the multiple pixels of the reference pixel block. For any reference pixel block, the electronic device obtains a second residual block of the reference pixel block by subtracting the values ​​of multiple pixels of the reference pixel block from the values ​​of multiple pixels of the corresponding second pixel block pixel by pixel. This second residual block represents the difference between the reference pixel block and the second pixel block. The pixel-by-pixel subtraction involves subtracting the values ​​of multiple pixels of the corresponding second pixel block from the values ​​of multiple pixels of the reference pixel block. The electronic device pre-encodes multiple second residual blocks of multiple reference pixel blocks. Based on the multiple pixel values ​​in any pre-encoded second residual block, the electronic device determines the mean square sum of the second residual blocks by squaring and summing the multiple pixel values. This mean square sum of the second residual blocks represents the quantization error between the reference pixel block and the second pixel block. In this embodiment, the electronic device can determine the average value of the mean square sums of the multiple second residual blocks as the quantization error between the filtered video frame and the reference video frame, i.e., the second quantization loss. Electronic devices can estimate the loss generated during the formal encoding of filtered video frames based on multiple second quantization losses. They can then determine whether to retain a filtered video frame by comparing the sum of the differences between multiple first quantization losses and multiple second quantization losses with the magnitude of the filtered quantization loss, thereby achieving higher video coding quality.

[0118] Optionally, the electronic device can determine the quantization error, i.e. the second quantization loss, between the filtered video frame and the reference video frame based on the mean square sum of multiple second residual blocks according to the following formula (3).

[0119]

[0120] Among them, quantFrameSSE(F n ,F i ) represents the filtered video frame F′ n With reference video frame F i The quantization error between the reference video frame F iWhen the quantity is 8, i = n-4, n-3, n-2, n-1, n+1, n+2, n+3, n+4; quantSSE k For the second residual block ΔB′ k The mean square sum, ΔB k For the k-th second residual block, ΔB′ k is the k-th second residual block after precoding; m is the number of second residual blocks.

[0121] It should be noted that the electronic device can determine whether to retain a filtered video frame by comparing the sum of multiple reference losses with the filtering quantization loss. If the sum of multiple reference losses is less than the filtering quantization loss, the electronic device performs step S308; if the sum of multiple reference losses is not less than the filtering quantization loss, the electronic device performs step S309.

[0122] In step S308, when the sum of multiple reference losses is less than the filtering quantization loss, the electronic device encodes the video frame. The reference loss is the difference between the corresponding first quantization loss and second quantization loss, and the multiple first quantization losses correspond one-to-one with the multiple second quantization losses.

[0123] In this embodiment, the electronic device determines multiple reference losses by determining the difference between the first quantization loss and the second quantization loss based on the correspondence between the first quantization loss and the second quantization loss. The multiple first quantization losses are estimates of the loss generated during the formal encoding of video frames, and the multiple second quantization losses are estimates of the loss generated during the formal encoding of filtered video frames. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the loss generated during the pre-encoding of video frames is less than the total loss generated during the pre-encoding of filtered video frames and the filtering of video frames. Therefore, it can be inferred that the loss generated during the formal encoding of video frames is less than the total loss generated during the formal encoding of filtered video frames and the filtering of video frames. This means that the electronic device generates less loss when encoding video frames, thus improving the encoding quality of video frames compared to encoding filtered video frames.

[0124] In step S309, the electronic device encodes the filtered video frame when the sum of the multiple reference losses is not less than the filtering quantization loss.

[0125] In this embodiment of the disclosure, when the sum of multiple reference losses is not less than the filtering quantization loss, it indicates that the loss generated during video frame precoding is not less than the total loss generated during precoding of the filtered video frame and during video frame filtering. Therefore, it can be inferred that the loss generated during formal encoding of the video frame is not less than the total loss generated during formal encoding of the filtered video frame and during video frame filtering. In this case, the loss generated when the electronic device encodes the filtered video frame is smaller, and the encoding quality of the video frame can be improved compared to encoding the video frame directly.

[0126] Optionally, the electronic device determines the sum of the differences between a plurality of first quantization losses and a plurality of second quantization losses according to the following formula (4).

[0127]

[0128] Wherein, refQuantSSE is multiple first quantization losses quantFrameSSE(F n ,F i ) and multiple second quantization losses quantFrameSSE(F′ n ,F i The sum of the differences between F and F; n For the current video frame; F′ n For filtered video frames; F i As a reference video frame, when the reference video frame F i When the quantity is 8, i = n-4, n-3, n-2, n-1, n+1, n+2, n+3, n+4. In refQuantSSE < α·quanFrameSSE(F n ,F′ n In the case where the sum of multiple reference losses is less than the filtering quantization loss, the electronic device does not retain the filtered video frame of the video frame and encodes the video frame; when refQuantSSE≥α·quanFrameSSE(F n ,F′ n In the case of ), it indicates that the sum of multiple reference losses is not less than the filtering quantization loss. The electronic device retains the filtered video frame of the video frame and encodes the filtered video frame.

[0129] This disclosure provides a video frame encoding method. By filtering a video frame, a filtered video frame and a filtering quantization loss can be obtained. Based on multiple reference video frames, the video frame and the filtered video frame are pre-encoded separately. Multiple first quantization losses generated during the video frame pre-encoding process and multiple second quantization losses generated during the filtered video frame pre-encoding process can be determined based on the pre-encoded video frame and the filtered video frame. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the loss generated during video frame pre-encoding is less than the total loss generated during filtering and pre-encoding of the filtered video frame. Therefore, it can be inferred that the loss generated during the formal encoding of the video frame is less than the total loss generated during filtering and pre-encoding of the filtered video frame. In this case, the electronic device can achieve a smaller loss during the formal encoding process, thus improving the video encoding quality. By comparing the sum of the differences between the multiple quantization losses generated during the pre-encoding process of the video frames before and after filtering with the magnitude of the filtering quantization loss, the electronic device can adaptively determine whether to retain the filtered video frame of the current video frame, thereby improving the video encoding quality.

[0130] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0131] Figure 4 This is a block diagram illustrating a video frame encoding apparatus according to an exemplary embodiment. (Refer to...) Figure 4 The device includes a filtering unit 401, a precoding unit 402, and an encoding unit 403.

[0132] The filtering unit 401 is configured to filter any video frame in the video frame queue based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame.

[0133] The precoding unit 402 is configured to precode a video frame and a filtered video frame based on multiple reference video frames to determine multiple first quantization losses and multiple second quantization losses. The multiple first quantization losses are used to indicate the quantization error between the video frame and the multiple reference video frames, and the multiple second quantization losses are used to indicate the quantization error between the second video frame and the multiple reference video frames, respectively. The multiple first quantization losses correspond one-to-one with the multiple second quantization losses.

[0134] The encoding unit 403 is configured to encode a video frame when the sum of multiple reference losses is less than the filtering quantization loss, wherein the reference loss is the difference between the corresponding first quantization loss and second quantization loss.

[0135] In some embodiments, Figure 5 This is a block diagram illustrating a video frame encoding apparatus according to an exemplary embodiment. See also: Figure 5 The filter unit 401 includes:

[0136] The filtering subunit 501 is configured to filter any video frame in the video frame queue based on multiple reference video frames to obtain a filtered video frame.

[0137] The first block subunit 502 is configured to divide the video frame into blocks to obtain multiple first pixel blocks;

[0138] The second block subunit 503 is configured to divide the filtered video frame into blocks to obtain multiple second pixel blocks, and the multiple second pixel blocks correspond one-to-one with multiple first pixel blocks.

[0139] The first determining subunit 504 is configured to determine the filtering quantization loss based on a plurality of first pixel blocks and a plurality of second pixel blocks.

[0140] In some embodiments, the first determining subunit 504 is configured to, for any first pixel block, subtract the corresponding second pixel block by pixels to obtain a residual block of the first pixel block; pre-encode multiple residual blocks of multiple first pixel blocks; for any pre-encoded residual block, obtain the mean square sum of the residual block based on the multiple pixel values ​​of the residual block; and determine the average value of the mean square sum of the multiple residual blocks as the filtering quantization loss.

[0141] In some embodiments, see continue to see Figure 5 The precoding unit 402 includes:

[0142] The third block subunit 505 is configured to divide the reference video frame into blocks for any reference video frame to obtain multiple reference pixel blocks, and the multiple reference pixel blocks correspond one-to-one with multiple first pixel blocks.

[0143] The second determining subunit 506 is configured to determine a first quantization loss between reference video frames and video frames based on a plurality of reference pixel blocks and a plurality of first pixel blocks.

[0144] The third determining subunit 507 is configured to determine a second quantization loss between a reference video frame and a filtered video frame based on a plurality of reference pixel blocks and a plurality of second pixel blocks.

[0145] In some embodiments, the second determining subunit 506 is configured to, for any reference pixel block, subtract the reference pixel block from the corresponding first pixel block to obtain a first residual block of the reference pixel block; pre-encode multiple first residual blocks of multiple reference pixel blocks; for any pre-encoded first residual block, determine the mean square sum of the first residual block based on multiple pixel values ​​of the first residual block; and determine the average value of the mean square sum of the multiple first residual blocks as a first quantization loss.

[0146] In some embodiments, the third determining subunit 507 is configured to, for any reference pixel block, subtract the reference pixel block from the corresponding second pixel block to obtain a second residual block of the reference pixel block; pre-encode multiple second residual blocks of multiple reference pixel blocks; for any pre-encoded second residual block, determine the mean square sum of the second residual block based on multiple pixel values ​​of the second residual block; and determine the average value of the mean square sum of the multiple second residual blocks as a second quantization loss.

[0147] In some embodiments, the encoding unit 403 is further configured to encode the filtered video frame if the sum of a plurality of reference losses is not less than the filtering quantization loss.

[0148] This disclosure provides a video frame encoding apparatus that, by filtering video frames, can obtain filtered video frames and filtering quantization losses. Based on multiple reference video frames, pre-encoding is performed on the video frame and the filtered video frame respectively. Multiple first quantization losses generated during the video frame pre-encoding process and multiple second quantization losses generated during the filtered video frame pre-encoding process can be determined based on the pre-encoded video frame and the filtered video frame. If the sum of the multiple reference losses is less than the filtering quantization loss, it indicates that the loss generated during video frame pre-encoding is less than the total loss generated during filtered video frame pre-encoding and video frame filtering. Therefore, it can be inferred that the loss generated during the formal encoding of the video frame is less than the total loss generated during filtered video frame formal encoding and video frame filtering. In this case, the electronic device can obtain less loss during the formal encoding process, thereby improving the video encoding quality. By comparing the sum of the differences between the multiple quantization losses generated during the pre-encoding process of the video frames before and after filtering with the magnitude of the filtering quantization loss, the electronic device can adaptively determine whether to retain the filtered video frame of the current video frame, thereby improving the video encoding quality.

[0149] It should be noted that the video frame encoding device provided in the above embodiments is only illustrated by the division of the above functional units when running the application. In actual applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the electronic device can be divided into different functional units to complete all or part of the functions described above. In addition, the video frame encoding device and the video frame encoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0150] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0151] When an electronic device is provided as a terminal, Figure 6 This is a block diagram illustrating a terminal 600 according to an exemplary embodiment. The terminal... Figure 6 A structural block diagram of a terminal 600 provided in an exemplary embodiment of this disclosure is shown. The terminal 600 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 600 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0152] Typically, terminal 600 includes a processor 601 and a memory 602.

[0153] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0154] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one program code, which is executed by the processor 601 to implement the video frame encoding method provided in the method embodiments of this disclosure.

[0155] In some embodiments, the terminal 600 may also optionally include a peripheral device interface 603 and at least one peripheral device. The processor 601, memory 602, and peripheral device interface 603 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 603 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608, and a power supply 606.

[0156] Peripheral interface 603 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 601 and memory 602. In some embodiments, processor 601, memory 602 and peripheral interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 601, memory 602 and peripheral interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0157] The radio frequency (RF) circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 604 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 604 may also include circuitry related to NFC (Near Field Communication), which is not limited in this disclosure.

[0158] Display screen 605 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 605 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 601 for processing. In this case, display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 605, which serves as the front panel of terminal 600; in other embodiments, there may be at least two display screens, respectively disposed on different surfaces of terminal 600 or in a folded design; in still other embodiments, display screen 605 may be a flexible display screen, disposed on a curved or folded surface of terminal 600. Furthermore, display screen 605 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 605 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0159] The camera assembly 606 is used to acquire images or videos. Optionally, the camera assembly 606 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0160] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 601 for processing, or input to the radio frequency circuit 604 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 600. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 607 may also include a headphone jack.

[0161] Power supply 608 is used to power the various components in terminal 600. Power supply 608 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 608 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0162] In some embodiments, the terminal 600 further includes one or more sensors 606. The one or more sensors 606 include, but are not limited to, an accelerometer 610, a gyroscope 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.

[0163] Accelerometer 610 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 600. For example, accelerometer 610 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 601 can control display screen 605 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 610. Accelerometer 610 can also be used for games or for acquiring user motion data.

[0164] The gyroscope sensor 611 can detect the orientation and rotation angle of the terminal 600. The gyroscope sensor 611 can work in conjunction with the accelerometer sensor 610 to collect the user's 3D movements on the terminal 600. Based on the data collected by the gyroscope sensor 611, the processor 601 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0165] The pressure sensor 612 can be disposed on the side bezel of the terminal 600 and / or on the lower layer of the display screen 605. When the pressure sensor 612 is disposed on the side bezel of the terminal 600, it can detect the user's grip signal on the terminal 600, and the processor 601 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612. When the pressure sensor 612 is disposed on the lower layer of the display screen 605, the processor 601 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 605. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0166] An optical sensor 613 is used to collect ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the display screen 605 based on the ambient light intensity collected by the optical sensor 613. Optionally, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is decreased. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 based on the ambient light intensity collected by the optical sensor 613.

[0167] A proximity sensor 614, also known as a distance sensor, is installed on the front panel of the terminal 600. The proximity sensor 614 is used to detect the distance between the user and the front of the terminal 600. In one embodiment, when the proximity sensor 614 detects that the distance between the user and the front of the terminal 600 is gradually decreasing, the processor 601 controls the display screen 605 to switch from a screen-on state to a screen-off state; when the proximity sensor 614 detects that the distance between the user and the front of the terminal 600 is gradually increasing, the processor 601 controls the display screen 605 to switch from a screen-off state to a screen-on state.

[0168] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on terminal 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0169] When electronic devices are provided as servers, Figure 7This is a block diagram illustrating a server 700 according to an exemplary embodiment. The server 700 can vary significantly due to differences in configuration or performance. It may include one or more Central Processing Units (CPUs) 701 and one or more memories 702. The memories 702 store at least one line of program code, which is loaded and executed by the processor 701 to implement the video frame encoding methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 700 may also include other components for implementing device functions, which will not be elaborated upon here.

[0170] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as memory 602 or memory 702 including instructions, which can be executed by processor 601 of terminal 600 or processor 701 of server 700 to perform the above method. Optionally, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0171] A computer program product includes a computer program / instructions that, when executed by a processor, implement the aforementioned video frame encoding method.

[0172] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0173] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video frame encoding method, characterized in that, The method includes: For any video frame in the video frame queue, the video frame is filtered based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame. Based on the plurality of reference video frames, the video frame and the filtered video frame are pre-coded respectively to determine a plurality of first quantization losses and a plurality of second quantization losses. The plurality of first quantization losses are used to indicate the quantization error between the video frame and the plurality of reference video frames respectively, and the plurality of second quantization losses are used to indicate the quantization error between the filtered video frame and the plurality of reference video frames respectively. The plurality of first quantization losses and the plurality of second quantization losses correspond one-to-one. The video frame is encoded when the sum of multiple reference losses is less than the product of the filtering quantization loss and the preset parameter, and the reference loss is the difference between the corresponding first quantization loss and the second quantization loss.

2. The video frame encoding method according to claim 1, characterized in that, For any video frame in the video frame queue, filtering is performed on the video frame based on multiple reference video frames to obtain the filtered video frame and the filtering quantization loss, including: For any video frame in the video frame queue, the video frame is filtered based on multiple reference video frames to obtain the filtered video frame; The video frame is divided into blocks to obtain multiple first pixel blocks; The filtered video frame is divided into blocks to obtain multiple second pixel blocks, and the multiple second pixel blocks correspond one-to-one with the multiple first pixel blocks; The filtering quantization loss is determined based on the plurality of first pixel blocks and the plurality of second pixel blocks.

3. The video frame encoding method according to claim 2, characterized in that, The step of determining the filtering quantization loss based on the plurality of first pixel blocks and the plurality of second pixel blocks includes: For any first pixel block, subtract the first pixel block from the corresponding second pixel block by pixels to obtain the residual block of the first pixel block; Pre-encode multiple residual blocks of the plurality of first pixel blocks; For any residual block after precoding, the mean square sum of the residual block is obtained based on multiple pixel values ​​of the residual block; The average value of the sum of the mean squares of the plurality of residual blocks is determined as the filtering quantization loss.

4. The video frame encoding method according to claim 2, characterized in that, The step of precoding the video frames and the filtered video frames based on the plurality of reference video frames to determine a plurality of first quantization losses and a plurality of second quantization losses includes: For any reference video frame, the reference video frame is divided into blocks to obtain multiple reference pixel blocks, and the multiple reference pixel blocks correspond one-to-one with the multiple first pixel blocks; Based on the plurality of reference pixel blocks and the plurality of first pixel blocks, a first quantization loss between the reference video frame and the video frame is determined; Based on the plurality of reference pixel blocks and the plurality of second pixel blocks, a second quantization loss is determined between the reference video frame and the filtered video frame.

5. The video frame encoding method according to claim 4, characterized in that, The step of determining the first quantization loss between the reference video frame and the video frame based on the plurality of reference pixel blocks and the plurality of first pixel blocks includes: For any reference pixel block, the reference pixel block and the corresponding first pixel block are subtracted by pixels to obtain the first residual block of the reference pixel block; Pre-encode multiple first residual blocks of the multiple reference pixel blocks; For any first residual block after precoding, the mean square sum of the first residual block is determined based on multiple pixel values ​​of the first residual block; The average value of the sum of the mean squares of the plurality of first residual blocks is determined as the first quantization loss.

6. The video frame encoding method according to claim 4, characterized in that, The step of determining the second quantization loss between the reference video frame and the filtered video frame based on the plurality of reference pixel blocks and the plurality of second pixel blocks includes: For any reference pixel block, the reference pixel block and the corresponding second pixel block are subtracted by pixels to obtain the second residual block of the reference pixel block; Pre-encode multiple second residual blocks of the plurality of reference pixel blocks; For any second residual block after precoding, the mean square sum of the second residual block is determined based on multiple pixel values ​​of the second residual block; The average value of the sum of the mean squares of the plurality of second residual blocks is determined as the second quantization loss.

7. The video frame encoding method according to claim 1, characterized in that, The method further includes: The filtered video frame is encoded if the sum of the plurality of reference losses is not less than the filtering quantization loss.

8. A video frame encoding apparatus, characterized in that, The device includes: The filtering unit is configured to filter any video frame in the video frame queue based on multiple reference video frames to obtain a filtered video frame and a filtering quantization loss. The reference video frames are used to provide reference information during the filtering process of the video frame, and the filtering quantization loss is used to indicate the quantization error between the video frame and the filtered video frame. The precoding unit is configured to precode the video frame and the filtered video frame based on the plurality of reference video frames to determine a plurality of first quantization losses and a plurality of second quantization losses. The plurality of first quantization losses are used to indicate the quantization error between the video frame and the plurality of reference video frames, and the plurality of second quantization losses are used to indicate the quantization error between the filtered video frame and the plurality of reference video frames, respectively. The plurality of first quantization losses and the plurality of second quantization losses correspond one-to-one. The encoding unit is configured to encode the video frame when the sum of a plurality of reference losses is less than the product of the filter quantization loss and a preset parameter, wherein the reference loss is the difference between the corresponding first quantization loss and the second quantization loss.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the video frame encoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the video frame encoding method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the video frame encoding method according to any one of claims 1 to 7.

Images