Video encoding method and corresponding apparatus

Abstract

The present application provides a video encoding method, which is applied to video encoding scenes. When a video sequence is encoded, the method comprises: on the basis of motion information of a first moving body in an encoded frame, predicting at least one target area in a frame to be encoded, wherein the at least one target area is related to the first moving body; and using a target quantization parameter to encode video data corresponding to the at least one target area, wherein the target quantization parameter is different from a global quantization parameter of the video sequence. According to the solution provided by the present application, during encoding, more encoding resources can be allocated to the target area having a small quantization parameter, and fewer encoding resources can be allocated to the target area having a large quantization parameter, so that the effectiveness of encoding resource allocation during video encoding can be improved, and the video encoding efficiency and quality are improved.

Description

A video encoding method and corresponding device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 28, 2023, with application number 202311849660.2 and application name “A method and corresponding device for video encoding”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of coding technology, and in particular to a video coding method and corresponding device. Background Art

[0003] Video encoding is a compression technology that can convert video files in one format into another. For example, video encoding can convert YUV video files into H.264 or other formats, thus achieving video compression.

[0004] During video encoding, the encoder encodes each video frame in the video sequence frame by frame according to the global quantization parameters. Video frames are continuous. If the relationship between video frames is not considered during video encoding, improper use of encoding resources will result. Summary of the Invention

[0005] The present application provides a video encoding method for improving the effectiveness of encoding resource allocation during video encoding, and also provides a corresponding apparatus, a computer-readable storage medium, and a computer program product.

[0006] A first aspect of the present application provides a method for video encoding. When encoding a video sequence, the method includes: predicting at least one target area in a pre-encoded frame based on motion information of a first moving body in an encoded frame, where the at least one target area is related to the first moving body; encoding video data corresponding to the at least one target area using a target quantization parameter, where the target quantization parameter is different from a global quantization parameter of the video sequence.

[0007] In the present application, the video sequence may be a YUV sequence. The encoded frame and the pre-encoded frame may be two consecutive video frames, or may be two video frames separated by one or more video frames.

[0008] In this application, a pre-encoded frame refers to a frame to be encoded or a frame to be encoded, usually referring to the next frame to be encoded after the previous frame has been encoded.

[0009] In the present application, the first moving object may be one moving object or multiple moving objects in the encoded frame. The at least one target area in the pre-encoded frame may be part or all of the area corresponding to the pre-encoded frame.

[0010] In the present application, the global quantization parameter is a quantization parameter (QP) configured for encoding a video sequence. In the prior art, each video frame in a video sequence needs to be encoded according to the global quantization parameter. However, in the present application, a target quantization parameter different from the global quantization parameter is used for encoding a target area related to the first moving body. The target quantization parameter can be greater than the global quantization parameter or less than the global quantization parameter. Moreover, when there are multiple target areas, the target quantization parameters corresponding to some target areas can be greater than the global quantization parameter, and the target quantization parameters corresponding to some target areas can be less than the global quantization parameter. During encoding, more encoding resources can be allocated to target areas with smaller quantization parameters, and fewer encoding resources can be allocated to target areas with larger quantization parameters. In this way, the effectiveness of encoding resource allocation during video encoding can be improved, and the efficiency and quality of video encoding can be improved.

[0011] In one possible implementation, when there is only one target area, the target area is the first area or the second area; when there are multiple target areas, the multiple target areas include the first area and the second area; wherein the first area is the moving area of ​​the first moving body in the pre-encoded frame, and the target quantization parameter corresponding to the first area is less than the global quantization parameter; the second area is the area in the pre-encoded frame, and the second area is occluded by the first moving body in the next frame of the pre-encoded frame, and the target quantization parameter corresponding to the second area is greater than the global quantization parameter.

[0012] In this possible implementation, the first region is the region where the first moving body is located in the predicted pre-encoded frame. Since high-quality display of the moving body is usually required during playback, a smaller global quantization parameter is used when encoding the video data corresponding to the first region. Allocating more encoding resources to the first region can improve the encoding quality of the first moving body in the pre-encoded frame. The second region is the region in the predicted pre-encoded frame that will be obscured by the first moving body in the next frame. Since the region will be obscured by the first moving body in the next frame of the pre-encoded frame, the high encoding quality of the second region in the pre-encoded frame is not of much value. Therefore, when encoding the video data corresponding to the second region, a larger global quantization parameter is used, and fewer encoding resources are allocated to the second region. This can reduce the waste of encoding resources and improve the utilization rate of encoding resources.

[0013] In a possible implementation, the encoded frame and the pre-encoded frame are consecutive frames or non-consecutive frames in a video sequence.

[0014] In this possible implementation, because object motion is continuous, if the coded frame and the pre-coded frame are two consecutive video frames, the accuracy of target region prediction in the pre-coded frame can be improved. Of course, if the coded frame and the pre-coded frame are two video frames separated by a certain distance, target region prediction in the pre-coded frame can also be achieved, thereby increasing the diversity of target region prediction in the pre-coded frame.

[0015] In one possible implementation, each video frame of a video sequence includes multiple blocks. The above step of predicting at least one target area in a pre-coded frame based on motion information of a first moving object in an encoded frame includes predicting position information of a motion block in the pre-coded frame based on a first-order motion vector of the motion block in the encoded frame, an interval between the encoded frame and the pre-coded frame, and position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is a block corresponding to the first moving object in the encoded frame, and the motion block in the pre-coded frame is a block corresponding to the predicted first moving object in the pre-coded frame; and determining the first area based on the position information of the motion block in the pre-coded frame.

[0016] In this application, a block can be a macroblock (MB) in H.264, or a block in each video frame can be a prediction unit (PU) in the H.265 scenario. Of course, the compression format of this application is not limited to H.264 and H.265, and can also be applied to other compression formats, such as H.266.

[0017] In the present application, the interval between the coded frame and the pre-coded frame may indicate the relationship between the motion vector of the j-th block in the pre-coded frame and the first-order motion vector of the j-th block in the coded frame.

[0018] In this possible implementation, between the encoded frame and the pre-encoded frame, the motion region of the first moving object in the pre-encoded frame, ie, the first region, is predicted according to the motion relationship between corresponding blocks between the two video frames, thereby improving the accuracy of the first region prediction.

[0019] In one possible implementation, each video frame of a video sequence includes multiple blocks. The above step of predicting at least one target area in a pre-coded frame based on motion information of a first moving body in an encoded frame includes predicting position information of an occluded block in the pre-coded frame based on a first-order motion vector of the motion block in the encoded frame, an interval between the encoded frame and the pre-coded frame, and position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is a block corresponding to the first moving body in the encoded frame, and the occluded block in the pre-coded frame is a block predicted to be occluded by the first moving body in a next frame of the pre-coded frame; and determining the second area based on the position information of the occluded block in the pre-coded frame.

[0020] In this possible implementation, between the encoded frame and the pre-encoded frame, the motion area of ​​the first moving body in the next frame of the pre-encoded frame is predicted according to the motion relationship of the corresponding blocks between the two video frames, thereby determining the occluded area in the pre-encoded frame, that is, the second area, which can improve the accuracy of the prediction of the second area.

[0021] In one possible implementation, the step of predicting the position information of the motion block in the pre-coded frame based on the first-order motion vector of the motion block in the coded frame, the interval between the coded frame and the pre-coded frame, and the position information of the motion block in the coded frame includes: determining the motion vector of the motion block in the pre-coded frame based on the first-order motion vector of the motion block in the coded frame and the interval between the coded frame and the pre-coded frame; wherein the interval between the coded frame and the pre-coded frame is used to indicate a multiple relationship between the motion vector and the first-order motion vector of the motion block in the pre-coded frame; and determining the position information of the motion block in the pre-coded frame based on the position information of the motion block in the coded frame and the motion vector of the motion block in the pre-coded frame.

[0022] In this possible implementation, in the present application, if the interval between the coded frame and the pre-coded frame is 0, the motion vector of the motion block in the pre-coded frame is twice the first-order motion vector; if the interval between the coded frame and the pre-coded frame is x, the motion vector of the motion block in the pre-coded frame is (2+x) times the first-order motion vector. In the present application, the position of the motion block in the pre-coded frame is determined by using the position information of the coded frame and the motion vector of the motion block in the pre-coded frame, which can improve the speed and accuracy of determining the position of the motion block in the pre-coded frame.

[0023] In one possible implementation, based on the first-order motion vector of the motion block in the coded frame, the interval between the coded frame and the precoded frame, and the position information of the motion block in the coded frame, the position information of the occluded block in the precoded frame is predicted, including: determining the motion vector of the motion block in the next frame of the precoded frame based on the first-order motion vector of the motion block in the coded frame and the interval between the coded frame and the precoded frame; wherein the interval between the coded frame and the precoded frame is used to indicate the multiple relationship between the motion vector and the first-order motion vector of the motion block in the next frame of the precoded frame; and determining the position information of the occluded block in the precoded frame based on the position information of the motion block in the coded frame and the motion vector of the motion block in the next frame of the precoded frame.

[0024] In this possible implementation, in the present application, if the interval between the coded frame and the pre-coded frame is 0, the motion vector of the motion block in the next frame of the pre-coded frame is three times the first-order motion vector; if the interval between the coded frame and the pre-coded frame is x, the motion vector of the motion block in the next frame of the pre-coded frame is (3+x) times the first-order motion vector. In the present application, the position of the occluded block in the pre-coded frame is determined by using the position information of the coded frame and the motion vector of the motion block in the next frame of the pre-coded frame, which can improve the speed and accuracy of determining the position of the occluded block in the pre-coded frame.

[0025] In one possible implementation, when there are multiple target areas, the method further includes: determining a first number of coordinate vectors of the motion area pointing to the first block based on the position information of the motion block in the pre-encoded frame, where the first block is any block in the encoded frame; determining a second number of coordinate vectors of the occluded area pointing to the first block based on the position information of the occluded block in the pre-encoded frame; and determining a quantization parameter of a second block corresponding to the first block in the pre-encoded frame based on the first number and the second number, where the quantization parameter of the second block is the target quantization parameter.

[0026] In this possible implementation, there may be multiple motion blocks surrounding the first block. Based on the previously determined position information of the motion blocks in the pre-coded frame, the number of coordinate vectors pointing to the motion region of the first block in the motion block corresponding to the first moving body, i.e., the first number, can be determined. Similarly, based on the position information of the occluded block in the pre-coded frame, the number of coordinate vectors pointing to the occluded region of the first block, i.e., the second number, can also be determined. By determining the quantization parameter of the second block in the pre-coded frame corresponding to the first block using the first number and the second number, the accuracy of the quantization parameter configuration of the second block can be improved, thereby improving the effectiveness of the coding resource allocation of the block.

[0027] In one possible implementation, the above steps of: determining the quantization parameter of the second block corresponding to the first block in the pre-coded frame based on the first quantity and the second quantity, include: determining the change in the quantization parameter of the second block based on the first quantity and the second quantity; determining the quantization parameter of the second block based on the change in the quantization parameter of the second block and the global quantization parameter.

[0028] In this possible implementation, the change in the quantization parameter is first determined by the first quantity and the second quantity, and then the quantization parameter of the second block is determined in combination with the global quantization parameter. This can improve the accuracy of the quantization parameter configuration of the second block, thereby improving the effectiveness of the coding resource allocation of the block.

[0029] In a possible implementation, determining the change amount of the quantization parameter of the second block according to the first quantity and the second quantity includes: determining the change amount of the quantization parameter of the second block according to the following relationship;

[0030] Among them, deltaQP j Indicates the change in the quantization parameter of the second block, A is a constant, ε is a constant, is the first quantity, The second quantity.

[0031] In this possible implementation, the speed and accuracy of determining the change in the quantization parameter of the second block can be improved by using the above relationship.

[0032] In one possible implementation, the method further includes: embedding a descriptor in the second block, the descriptor being used to indicate that the second block is a motion block or an occluded block, and when the second block is a motion block, the target quantization parameter used for encoding the second block is less than the global quantization parameter; when the second block is an occluded block, the target quantization parameter used for encoding the second block is greater than the global quantization parameter.

[0033] In this possible implementation, the descriptor can be 0 or 1, or other forms of indication, for example: 0 to indicate that the second block is a motion block, and 1 to indicate that the second block is an occluded block; or 1 to indicate that the second block is a motion block, and 0 to indicate that the second block is an occluded block. This application does not limit this. Indicating the target quantization parameter to be used when encoding the second block using the descriptor can improve the rationality of the allocation of coding resources for the second block.

[0034] In one possible implementation, in a hard encoder scenario, the method further includes:

[0035] Embed deltaQP in the second block j , deltaQP j Used to determine the corresponding target quantization parameter when encoding the second block.

[0036] In this possible implementation, deltaQP is embedded in the second block. j , which can improve the rationality of the allocation of coding resources for the second block.

[0037] In a possible implementation, the video sequence is included in a decoded video stream.

[0038] In this possible implementation, the decoded video stream may be encoded according to the above-mentioned video encoding process, thereby achieving secondary compression of the video data.

[0039] A second aspect of the present application provides a video encoding apparatus. When encoding a video sequence, the apparatus includes:

[0040] a prediction unit, configured to predict at least one target region in the pre-encoded frame based on motion information of the first moving object in the encoded frame, wherein the at least one target region is related to the first moving object;

[0041] The encoding unit is configured to encode video data corresponding to at least one target region using a target quantization parameter, where the target quantization parameter is different from a global quantization parameter of the video sequence.

[0042] In one possible implementation, when there is only one target area, the target area is the first area or the second area; when there are multiple target areas, the multiple target areas include the first area and the second area; wherein the first area is the moving area of ​​the first moving body in the pre-encoded frame, and the target quantization parameter corresponding to the first area is less than the global quantization parameter; the second area is the area in the pre-encoded frame, and the second area is occluded by the first moving body in the next frame of the pre-encoded frame, and the target quantization parameter corresponding to the second area is greater than the global quantization parameter.

[0043] In a possible implementation, the encoded frame and the pre-encoded frame are consecutive frames or non-consecutive frames in a video sequence.

[0044] In one possible implementation, the prediction unit is specifically configured to:

[0045] When each video frame of the video sequence includes multiple blocks, position information of the motion block in the pre-coded frame is predicted based on a first-order motion vector of the motion block in the coded frame, an interval between the coded frame and the pre-coded frame, and position information of the motion block in the coded frame; wherein the motion block in the coded frame is a block corresponding to a first moving object in the coded frame, and the motion block in the pre-coded frame is a block corresponding to the predicted first moving object in the pre-coded frame;

[0046] The first region is determined according to position information of the motion block in the pre-encoded frame.

[0047] In one possible implementation, the prediction unit is specifically configured to:

[0048] When each video frame of a video sequence includes multiple blocks, position information of an occluded block in a pre-coded frame is predicted based on a first-order motion vector of a motion block in an encoded frame, an interval between the encoded frame and a pre-coded frame, and position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is a block corresponding to a first moving object in the encoded frame, and the occluded block in the pre-coded frame is a block predicted to be occluded by the first moving object in a frame next to the pre-coded frame;

[0049] The second area is determined according to the position information of the occluded block in the pre-encoded frame.

[0050] In one possible implementation, the prediction unit is specifically configured to:

[0051] Determining a motion vector of a motion block in a pre-coded frame based on a first-order motion vector of the motion block in the coded frame and an interval between the coded frame and the pre-coded frame; wherein the interval between the coded frame and the pre-coded frame is used to indicate a multiple relationship between the motion vector of the motion block in the pre-coded frame and the first-order motion vector;

[0052] The position information of the motion block in the pre-encoded frame is determined according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the pre-encoded frame.

[0053] In one possible implementation, the prediction unit is specifically configured to:

[0054] Determining, based on a first-order motion vector of a motion block in a coded frame and an interval between the coded frame and the pre-coded frame, a motion vector of a motion block in a frame next to the pre-coded frame; wherein the interval between the coded frame and the pre-coded frame is used to indicate a multiple relationship between the motion vector of the motion block in the frame next to the pre-coded frame and the first-order motion vector;

[0055] The position information of the blocked block in the pre-encoded frame is determined according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the next frame of the pre-encoded frame.

[0056] In a possible implementation, the prediction unit is further configured to: when there are multiple target areas;

[0057] determining, based on position information of the motion block in the pre-encoded frame, a first number of coordinate vectors pointing to a motion region of a first block, where the first block is any block in the encoded frame;

[0058] determining, according to position information of the occluded block in the pre-encoded frame, a second number of coordinate vectors pointing to the occluded area of ​​the first block;

[0059] A quantization parameter of a second block corresponding to the first block in the pre-encoded frame is determined according to the first quantity and the second quantity, and the quantization parameter of the second block is a target quantization parameter.

[0060] In one possible implementation, the prediction unit is specifically configured to:

[0061] determining a change amount of a quantization parameter of the second block according to the first amount and the second amount;

[0062] The quantization parameter of the second block is determined according to the change amount of the quantization parameter of the second block and the global quantization parameter.

[0063] In one possible implementation, the prediction unit is specifically configured to:

[0064] Determine the change in the quantization parameter of the second block according to the following relationship:

[0065] Among them, deltaQP j Indicates the change in the quantization parameter of the second block, A is a constant, ε is a constant, is the first quantity, The second quantity.

[0066] In one possible implementation, the prediction unit is further configured to:

[0067] A descriptor is embedded in the second block, where the descriptor is used to indicate that the second block is a motion block or an occluded block. When the second block is a motion block, a target quantization parameter used for encoding the second block is less than a global quantization parameter. When the second block is an occluded block, a target quantization parameter used for encoding the second block is greater than the global quantization parameter.

[0068] In one possible implementation, in a hard encoder scenario, the prediction unit is further configured to: embed deltaQP in the second block j , deltaQP j Used to determine the corresponding target quantization parameter when encoding the second block.

[0069] In a possible implementation, the video sequence is included in a decoded video stream.

[0070] In a possible implementation, the blocks in each video frame are macroblocks MB in an H.264 scenario, or the blocks in each video frame are prediction units PU in an H.265 scenario.

[0071] A third aspect of the present application provides a video encoding apparatus, which has the functionality to implement the video encoding method of the first aspect or any possible implementation of the first aspect. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functionality, such as a prediction unit and an encoding unit.

[0072] In a fourth aspect, the present application provides an electronic device comprising a communication interface, an encoder, a processor and a memory. The communication interface and the encoder, the processor and the memory are coupled. The memory is used to store programs or instructions. When the programs or instructions are executed by the processor, the electronic device executes the method as described in the first aspect or any possible implementation of the first aspect.

[0073] In the present application, the processor may include at least one of a central processing unit (CPU) and a graphics processing unit (GPU); wherein both the CPU and the GPU may execute the video encoding process described in the above-mentioned first aspect or any possible implementation of the first aspect, or the CPU and the GPU cooperate to execute the video encoding process described in the above-mentioned first aspect or any possible implementation of the first aspect.

[0074] In a fifth aspect, the present application provides a chip system comprising one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from a memory of an electronic device and send signals to the processor, the signals comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes a method as described in the first aspect or any possible implementation of the first aspect, and the processor is at least one of a CPU and a GPU.

[0075] In a sixth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any possible implementation of the first aspect.

[0076] In a seventh aspect, the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer executes the method of the first aspect or any possible implementation of the first aspect.

[0077] In an eighth aspect, the present application provides a video processing system, including: an electronic device, the electronic device being used to execute the method of the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0078] In a ninth aspect, the present application provides a device for storing a code stream, comprising at least one storage medium and a communication interface; the communication interface is used to receive or send the code stream; the at least one storage medium is used to store the code stream; the code stream is encoded by an encoder executing the method of the above-mentioned first aspect or any possible implementation method of the first aspect.

[0079] In a tenth aspect, the present application provides a method for storing a code stream, comprising: receiving a code stream through a communication interface; storing the code stream in one or more storage media, wherein the code stream is encoded by an encoder executing the method of the first aspect or any possible implementation method of the first aspect.

[0080] In an eleventh aspect, the present application provides a system for distributing a bitstream, comprising at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, where the bitstream is encoded by an encoder executing the method of the first aspect or any possible implementation of the first aspect;

[0081] The video stream device is used for sending a target code stream in at least one storage medium to the decoder in response to a request from the decoder.

[0082] In a twelfth aspect of the present application, a method for distributing a code stream is provided, comprising: receiving a first request; selecting a target code stream from at least one storage medium in response to the first request; and sending the target code stream to a destination device; the at least one storage medium is used to store the code stream, wherein the target code stream is encoded by an encoder executing the method of the first aspect or any possible implementation of the first aspect.

[0083] A thirteenth aspect of the present application provides a system for processing a code stream, including an image source device, an encoder device, one or more storage media, and a destination device;

[0084] The image source device is used to provide image data;

[0085] The encoder device is configured to obtain image data from an image source device through an interface and encode the image data to obtain one or more code streams, where the code streams are encoded by the encoder device by executing the method of the first aspect or any possible implementation of the first aspect.

[0086] The encoder device is used to store one or more code streams into one or more storage media; or the encoder device is used to encapsulate one or more code streams to obtain a transmission code stream;

[0087] The encoder device is used to transmit the transport stream to the destination device through a communication link or a communication network; the destination device is used to decapsulate the transport stream to obtain one or more code streams;

[0088] The destination device is used to decode one or more code streams to obtain decoded data.

[0089] The relevant features and effects of the second aspect of this application, as well as any possible implementation of the second aspect to the thirteenth aspect, can be understood by referring to the corresponding introduction in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] FIG1A is a schematic diagram of a scenario architecture provided by an embodiment of the present application;

[0091] FIG1B is a schematic diagram of another scenario architecture provided by an embodiment of the present application;

[0092] FIG2 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0093] FIG3 is a schematic diagram of an embodiment of a video encoding method provided in an embodiment of the present application;

[0094] FIG4 is a schematic diagram of an example scenario provided by an embodiment of the present application;

[0095] FIG5A is a schematic diagram of another embodiment of a video encoding method provided in an embodiment of the present application;

[0096] FIG5B is a schematic diagram illustrating an example of regional prediction provided by an embodiment of the present application;

[0097] FIG6 is a schematic diagram of another embodiment of a video encoding method provided in an embodiment of the present application;

[0098] FIG7 is a structural diagram of a video encoding apparatus provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0100] The terms "first," "second," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0101] The present application provides a video encoding method for improving the effectiveness of encoding resource allocation during video encoding. The present application also provides a corresponding apparatus, a computer-readable storage medium, and a computer program product. These are described in detail below.

[0102] To facilitate understanding, the following briefly introduces the technical terms involved in the embodiments of this application:

[0103] YUV format: yuv format is a picture format, which consists of three parts: Y, U, and V. Y represents brightness, and U and V represent color chromaticity.

[0104] H.264: H.264 is a new generation of digital video compression format jointly proposed by the International Organization for Standardization and the International Telecommunication Union. It is one of the video coding and decoding technology standards named in the H.26x series.

[0105] H.265: H.265 is also one of the video codec technology standards named in the H.26x series.

[0106] Video stream or video sequence: consists of multiple video frames, which can include I frames, P frames and B frames.

[0107] I frame: also known as a complete frame or key frame. The contents of the I frame need to be saved during encoding.

[0108] P-frames: Also called forward reference frames, they use inter-frame compression technology. P-frames only need to store data that is different from the previous frame, and the previous frame is referenced during compression.

[0109] B frame: also called bidirectional reference frame, it uses inter-frame compression technology and refers to both the previous frame and the next frame during compression.

[0110] Each video frame can be further divided into slices, and slices can be further divided into blocks.

[0111] Slices are the carriers of macroblocks. Slices are designed to limit the propagation and transmission of errors. Coded slices are project-independent; prediction for one slice cannot use macroblocks in other slices as a reference image. This ensures that prediction errors in one slice do not propagate to other slices. An image can consist of one or more slices, each containing an integer number of macroblocks. Each slice contains at least one macroblock, and at most, a slice contains the entire image's macroblocks. Slices can be subdivided into a "slice header + slice data." Because a frame of data may not be fully transmitted in one go, header information is required.

[0112] Macroblocks (MBs): These are the primary carriers of video information, containing the luminance and chrominance information for each pixel. A macroblock typically consists of a 16×16 luminance pixel block and an additional 8×8 Cb and 8×8 Cr color pixel block. Within each image, several macroblocks are arranged into slices. A macroblock contains information such as the macroblock type, prediction type, coded block pattern, quantization parameter (QP), and the pixel's luminance and chrominance data sets. MB is typically the term used in H.264.

[0113] In H.265, the specific division method and name of video frames are slightly different from H.264. In H.265, the prediction unit (PU) corresponds to the MB.

[0114] The video encoding method provided in the embodiment of the present application can be applied to various scenarios that require video encoding, and is particularly suitable for scenarios with moving objects. Taking the smart traffic scenario as an example, the solution provided in the embodiment of the present application can encode the video shot by the camera (also called a camera) and then send it to the device that stores the code stream. The device that stores the code stream can be a local storage device or a cloud device. The encoder for encoding the video shot by the camera can be configured on the camera, or it can be configured on a near-end device separated from the camera, and the near-end device is usually installed at a position close to the camera.

[0115] The scenario where the encoder is configured on the camera can be understood by referring to Figure 1A. As shown in Figure 1A, taking an intersection as an example, there are four cameras at the intersection. Each camera can shoot the traffic conditions within a certain range of the intersection. The video data shot by each camera will enter the encoder of the camera in the form of a video sequence. The encoder will encode each video frame in the video sequence frame by frame, and then output the encoded video stream. As shown in Figure 1A, the encoded video streams output by the four cameras are encoded video stream 1, encoded video stream 2, encoded video stream 3, and encoded video stream 4. Each camera can send the encoded video stream to the cloud storage through the network, or send the encoded video stream to a local storage device for storage. This application does not limit this.

[0116] The scenario in which the encoder is configured on the near-end device of the camera can be understood by referring to Figure 1B. As shown in Figure 1B, taking an intersection as an example, there are four cameras at the intersection, and each camera can shoot the traffic conditions within a certain range of the intersection. The video data shot by each camera will first be transmitted to the near-end device, and the encoder configured on the near-end device will encode the video data transmitted by each camera respectively. It should be noted that one or more encoders can be configured on the near-end device. If one encoder is configured, the encoder needs to encode the video data transmitted by multiple cameras respectively. If multiple encoders are configured, different encoders can encode the video data transmitted by different cameras. After the near-end device encodes the video data of the four cameras, four encoded video streams can be obtained, namely: encoded video stream 1, encoded video stream 2, encoded video stream 3, and encoded video stream 4. The near-end device can send the encoded video stream to the cloud storage through the network, or it can send the encoded video stream to the local storage device for storage. This application does not limit this.

[0117] It should be noted that the encoding scheme provided in the embodiment of the present application can also be encoded in the cloud. In this case, the camera transmits the captured video data to the cloud through the network, and the cloud encodes the video data of each camera to obtain the corresponding encoded video stream.

[0118] In the video encoding scenario described in FIG. 1A or FIG. 1B above, whether encoding is performed by a camera, a proximal device, or a cloud device, the structures of these electronic devices that may complete the video encoding provided in the embodiments of the present application can be understood by referring to FIG. 2 .

[0119] Figure 2 is a schematic diagram of a possible logical structure of a cloud device / roadside device provided in an embodiment of the present application. As shown in Figure 2, the electronic device 20 provided in an embodiment of the present application includes: an encoder 200, a processor 201, a communication interface 202, a memory 203 and a bus 204. The encoder 200, the processor 201, the communication interface 202 and the memory 203 are interconnected via the bus 204. In an embodiment of the present application, the processor 201 is used to control and manage the actions of the electronic device 20, and the encoder 200 is used to encode the video data acquired by the electronic device 20. The communication interface 202 is used to support the electronic device 20 to communicate, for example: the communication interface 202 can acquire video data and send encoded video data. The memory 203 is used to store the program code and data of the electronic device 20.

[0120] The processor 201 may be a central processing unit (CPU), a general-purpose processor (GPOR), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device (PLD), a transistor logic device (TLD), a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. A processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The bus 204 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, for example. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG. 2 shows only one thick line, but this does not imply that there is only one bus or only one type of bus.

[0121] The following describes a video encoding method provided in an embodiment of the present application. The method can be executed by an electronic device or by a component of the electronic device (eg, an encoder, a processor, a chip, or a chip system).

[0122] As shown in FIG3 , an embodiment of a video encoding method provided by an embodiment of the present application includes:

[0123] 301. Predict at least one target area in a pre-encoded frame based on motion information of a first moving object in an encoded frame, where the at least one target area is related to the first moving object.

[0124] In the present application, the video sequence may be a YUV sequence. The encoded frame and the pre-encoded frame may be two consecutive video frames, or may be two video frames separated by one or more video frames.

[0125] In this application, a pre-encoded frame refers to a frame to be encoded or a frame to be encoded, usually referring to the next frame to be encoded after the previous frame has been encoded.

[0126] In the present application, the first moving object may be one moving object or multiple moving objects in the encoded frame. The at least one target area in the pre-encoded frame may be part or all of the area corresponding to the pre-encoded frame.

[0127] 302. Encode video data corresponding to at least one target region using a target quantization parameter, where the target quantization parameter is different from a global quantization parameter of the video sequence.

[0128] In this application, the global quantization parameter is the quantization parameter configured for the encoded video sequence.

[0129] The solution provided by the embodiment of the present application is to encode the target area related to the first moving body using a target quantization parameter different from the global quantization parameter. The target quantization parameter can be greater than the global quantization parameter or less than the global quantization parameter. Moreover, when there are multiple target areas, the target quantization parameters corresponding to some target areas can be greater than the global quantization parameter, while the target quantization parameters corresponding to some target areas can be less than the global quantization parameter. During encoding, more encoding resources can be allocated to target areas with smaller quantization parameters, and fewer encoding resources can be allocated to target areas with larger quantization parameters. In this way, compared with the prior art in which each video frame in the video sequence needs to be encoded according to the global quantization parameter, the effectiveness of encoding resource allocation during video encoding can be improved, and the efficiency and quality of video encoding can be improved.

[0130] Optionally, in an embodiment of the present application, when there is only one target region, the target region may be the first region or the second region; when there are multiple target regions, the multiple target regions include the first region and the second region. The first region is the moving region of the first moving body in the pre-encoded frame, and the target quantization parameter corresponding to the first region is less than the global quantization parameter; the second region is the region in the pre-encoded frame, and the second region is occluded by the first moving body in the next frame of the pre-encoded frame, and the target quantization parameter corresponding to the second region is greater than the global quantization parameter.

[0131] In the embodiment of the present application, the coded frame and the pre-coded frame can be consecutive frames or non-consecutive frames in the video sequence. When the coded frame and the pre-coded frame are consecutive frames, the interval between the coded frame and the pre-coded frame is 0. When the coded frame and the pre-coded frame are non-consecutive frames, the interval between the coded frame and the pre-coded frame is the number of video frames between them.

[0132] Regarding the moving area and the blocked area, the three consecutive video frames in Figure 4 are used as an example. As shown in Figure 4, in the n-1th frame, the area where the first moving body is located is the moving area, and the blocked area in the n-1th frame is the area where the first moving body is predicted to move in the nth frame; similarly, in the nth frame, the area where the first moving body is located is the moving area, and the blocked area in the nth frame is the area where the first moving body is predicted to move in the n+1th frame. Within each frame, the motion vectors of the moving area and the blocked area can be In the n-1th frame, the motion vectors of the moving area and the occluded area are expressed as In the nth frame, the motion vectors of the moving area and the occluded area are expressed as In the n+1th frame, the motion vectors of the moving area and the occluded area are expressed as Between frames, because three video frames are continuous, That is to say, if the interval between the coded frame and the precoded frame is 0, the motion vector of the motion block in the precoded frame is twice the first-order motion vector of the corresponding motion block in the coded frame, and the motion vector of the motion block in the next frame of the precoded frame is three times the first-order motion vector of the corresponding motion block in the coded frame.

[0133] FIG4 shows three consecutive frames. If the interval between the coded frame and the precoded frame is x, where x is a positive integer, the motion vector of the motion block in the precoded frame is (2+x) times the first-order motion vector of the corresponding motion block in the coded frame, and the motion vector of the motion block in the next frame of the precoded frame is (3+x) times the first-order motion vector of the corresponding motion block in the coded frame.

[0134] In an embodiment of the present application, the process of predicting the first area (that is, the motion area) may be: predicting the position information of the motion block in the pre-coded frame based on the first-order motion vector of the motion block in the coded frame, the interval between the coded frame and the pre-coded frame, and the position information of the motion block in the coded frame; wherein the motion block in the coded frame is a block corresponding to the first moving body in the coded frame, and the motion block in the pre-coded frame is a block corresponding to the predicted first moving body in the pre-coded frame; and determining the first area based on the position information of the motion block in the pre-coded frame.

[0135] In an embodiment of the present application, the process of predicting the second area (that is, the occluded area) can be: based on the first-order motion vector of the motion block in the encoded frame, the interval between the encoded frame and the pre-encoded frame, and the position information of the motion block in the encoded frame, predict the position information of the occluded block in the pre-encoded frame; wherein, the motion block in the encoded frame is a block corresponding to the first moving body in the encoded frame, and the occluded block in the pre-encoded frame is a block predicted to be occluded by the first moving body in the next frame of the pre-encoded frame; and the second area is determined based on the position information of the occluded block in the pre-encoded frame.

[0136] In the embodiment of the present application, the process of determining the first area and the second area can also be understood by referring to FIG. 5A or FIG. 6 .

[0137] As shown in FIG5A , the coded frame and the precoded frame are consecutive frames. The coded frame can be represented by the n-1th frame, the precoded frame can be represented by the nth frame, and the next frame of the precoded frame can be represented by the n+1th frame, where n is an integer greater than 1.

[0138] 501. Obtain the first-order motion vector and position information of the i-th block in the n-1-th frame.

[0139] The first-order motion vector of the i-th block in the n-1-th frame can be Indicates that the location information of the block can be used express.

[0140] 502. Determine the motion vector and position information of the i-th block in the n-th frame and the motion vector and position information of the i-th block in the n+1-th frame according to the first-order motion vector of the i-th block in the n-1-th frame.

[0141] The running vector of the i-th block in the n-th frame can be expressed as Location information can be used Indicates. Because in the embodiment of the present application, the coded frame and the pre-coded frame are continuous frames, the interval between the coded frame and the pre-coded frame is 0, so Then get the position of the corresponding block in the nth frame

[0142] The running vector of the i-th block in the n+1th frame can be expressed as Location information can be used express. Then get the position of the corresponding block in the n+1th frame That is, the information of the block of the occluded area in the nth frame.

[0143] If there are M blocks in each frame, repeating steps 401 and 402 above can obtain the position information set of M blocks in the nth frame, which can be expressed as as well as in, Represents the predicted positions of all motion blocks in the nth frame, Indicates that the positions of all occluded blocks in the nth frame are predicted.

[0144] pass and The first area and the second area in the nth frame can be determined.

[0145] For an example of predicting moving and occluded areas in the nth frame, refer to Figure 5B. As shown in Figure 5B, the motion region prediction method uses the motion vector field of the coded (n-1)th frame, i.e., the first-order motion vector field, to predict the motion vector field of the current, uncoded, nth frame, i.e., the second-order motion vector field. The second-order motion vector field describes the motion characteristics of the current, uncoded frame, i.e., the nth frame. The number of second-order motion vectors pointing to the current block is used to predict the intensity of motion in the current block and identify the moving region. Taking block b1 in the n-1th frame as an example, the number M1 of second-order motion vectors pointing to block b1 is counted block by block. The larger M1, the higher the likelihood of a moving region in the next frame, and a smaller QP can be used for encoding block b1 in the nth frame.

[0146] Occluded area prediction: The motion vector field of the encoded (n-1)th frame, i.e., the first-order motion vector field, is used to predict the motion vector field of the next uncoded (n+1)th frame, i.e., the third-order motion vector field. The third-order motion vector field describes the motion characteristics and motion area of ​​the next uncoded frame, i.e., the (n+1)th frame. Using the reference relationship, it corresponds to the occluded area of ​​the currently encoded frame, i.e., the nth frame. The number of third-order motion vectors pointing to the current block is used to predict whether the current block will be occluded in the next frame. For example, the number M2 of third-order motion vectors pointing to the b2-b4 blocks is counted block by block. The larger M2, the higher the probability of occlusion in the next frame. Therefore, a larger QP is required for encoding the b2-b4 blocks in the nth frame.

[0147] 503. Determine a first quantity based on the position information of the motion block in the pre-encoded frame, and determine a second quantity based on the position information of the occluded block in the pre-encoded frame.

[0148] The first number is the number of coordinate vectors pointing to the motion area of ​​the first block, the second number is the number of coordinate vectors pointing to the blocked area of ​​the first block, and the first block is any block in the encoded frame.

[0149] Taking the jth block in the n-1th frame as an example, we can Count the number of moving object coordinate vectors pointing to the macroblock Can be based on Count the number of coordinate vectors of the occluded area pointing to the macroblock

[0150] According to the n-1 frame After that, you can Passed to the nth frame.

[0151] 504. Determine a quantization parameter of a second block corresponding to the first block in the pre-encoded frame according to the first quantity and the second quantity, the quantization parameter of the second block being a target quantization parameter.

[0152] Step 504 may include: determining a change in the quantization parameter of the second block according to the first quantity and the second quantity; and determining the quantization parameter of the second block according to the change in the quantization parameter of the second block and the global quantization parameter.

[0153] The second block is the jth block in the nth frame, and the change in the quantization parameter of the second block can be determined by the following relationship:

[0154] Among them, deltaQP j Indicates the change in the quantization parameter of the second block, A is a constant, ε is a constant, is the first quantity, The second quantity.

[0155] If the global quantization parameter is 23, the QP of the jth block in the nth frame is j =23+deltaQP j , the QP j That is the target quantization parameter of the jth block in the nth frame.

[0156] when Greater than When deltaQP j is a negative number, QP j It will be smaller than the global quantization parameter 23, that is, a smaller QP is used for high motion areas j Encoding; when Less than When deltaQP j is a positive number, QPj It will be greater than the global quantization parameter 23, that is, a larger QP is used for the occluded area j Encoding. When equal When deltaQP j 0, that is, QP j Equal to the global quantization parameter 23.

[0157] 505.Use QP j Encode the jth block in the nth frame and compare it with QP j The relevant information is embedded in the code stream of the jth block.

[0158] With QP j The relevant information may be a descriptor, which is used to indicate whether the jth block is a motion block or an occluded block, and when the jth block is a motion block, the target quantization parameter used to encode the jth block is less than the global quantization parameter, and when the jth block is an occluded block, the target quantization parameter used to encode the second block is greater than the global quantization parameter.

[0159] The descriptor can be 0 or 1, or other forms of indication, for example: 0 to indicate that the second block is a motion block, and 1 to indicate that the second block is an occluded block; or 1 to indicate that the second block is a motion block, and 0 to indicate that the second block is an occluded block. This application does not limit this. Indicating the target quantization parameter to be used when encoding the second block using the descriptor can improve the rationality of the allocation of coding resources for the second block.

[0160] Specifically, it can be: The descriptor representing the motion can be embedded into the bitstream of the jth block for subsequent decoding. It means that the jth block is an occluded descriptor, so as to indicate that the jth block is an occluded block, and the descriptor of the occluded block is embedded in the code stream.

[0161] Repeat the above operation for each block in the nth frame to complete the encoding of the nth frame.

[0162] By repeating the above process for each video frame in the video sequence, the encoding of all frames in the video sequence can be completed.

[0163] It should be noted that the process described in the above embodiment can be applied to the H.264 encoder. A YUV sequence is input to the H.264 encoder. The resolution of the sequence can be 1920x1080, the frame rate is 30fps, the IPPP prediction structure, and the global QP can be set to 23. Through the video encoding method provided above, each frame in the YUV sequence can be encoded.

[0164] If in the hard encoder scenario, and QP jThe relevant information can be deltaQP j , deltaQP j Embedded in the code stream of block j for subsequent decoding.

[0165] The above-mentioned video encoding scheme provided by the embodiment of the present application can also be applied to the scenario of secondary compression. The compressed video data can be decoded first, and the decoded YUV sequence can be input into the encoder to execute the video encoding process provided by the embodiment of the present application. For example, the input is an H.264 standard code stream v, and it is decoded into a YUV sequence of 300 frames in total. According to the GOP length of 60 frames, it is divided into 5 sub-video sequences, v1-v5. For v1-v5, the global QP can be set to 23. During the secondary encoding process, the v1 sub-video sequence can be encoded first to obtain the secondary encoded code stream v'1, and then the v2-v5 video sequences can be encoded one by one to obtain the secondary encoded code streams v'2~v'5, which are merged with v'1 to obtain the final secondary compressed code stream v'.

[0166] The above-mentioned encoding scheme provided in the embodiment of the present application can also be applied to the H.265 encoder. The process of applying to the H.265 encoder is the same as the above process, except that the block in the previous frame can be a macroblock (MB) in H.264, and in the H.265 encoder, the block in the previous frame can be a prediction unit (PU). Of course, the compression format of the present application is not limited to H.264 and H.265, and can also be applied to other compression formats, such as: H.266, etc.

[0167] In the embodiment of the present application, since the motion of an object is continuous, if the encoded frame and the pre-encoded frame are two consecutive video frames, the accuracy of the target area prediction in the pre-encoded frame can be improved.

[0168] FIG5A describes a scenario where the coded frame and the pre-coded frame are continuous frames. Referring to FIG6 below, a scenario where the coded frame and the pre-coded frame are non-continuous frames is described. Taking the interval between the coded frame and the pre-coded frame as an example of x video frames, where x is a positive integer, the coded frame can be represented by the m-1th frame, the pre-coded frame can be represented by the m+xth frame, and the next frame of the pre-coded frame can be represented by the m+x+1th frame.

[0169] 601. Obtain the first-order motion vector and position information of the i-th block in the m-1-th frame.

[0170] The first-order motion vector of the i-th block in the m-1-th frame can be Indicates that the location information of the block can be used express.

[0171] 602. Determine the motion vector and position information of the i-th block in the m+x-th frame and the motion vector and position information of the i-th block in the m+x+1-th frame based on the first-order motion vector of the i-th block in the m-1-th frame.

[0172] The running vector of the i-th block of the m+x-th frame can be expressed as Location information can be used Indicates that. Because in the embodiment of the present application, the interval between the coded frame and the pre-coded frame is x, so Then get the position of the corresponding block in the m+x frame

[0173] The running vector of the i-th block of the m+x+1-th frame can be expressed as Location information can be used Indicates. Then get the position of the corresponding block in the m+x+1th frame That is, the information of the block of the occluded area in the m+xth frame.

[0174] If there are M blocks in each frame, repeating steps 601 and 602 above can obtain the position information set of M blocks in the m+xth frame, which can be expressed as as well as in, Represents the predicted positions of all motion blocks in the m+xth frame, Indicates the predicted positions of all occluded blocks in the m+xth frame.

[0175] pass and The first area and the second area in the (m+x)th frame can be determined.

[0176] 603. Determine a first quantity based on the position information of the motion block in the pre-encoded frame, and determine a second quantity based on the position information of the occluded block in the pre-encoded frame.

[0177] The first number is the number of coordinate vectors pointing to the motion area of ​​the first block, the second number is the number of coordinate vectors pointing to the blocked area of ​​the first block, and the first block is any block in the encoded frame.

[0178] Taking the jth block in the m-1th frame as an example, we can Count the number of moving object coordinate vectors pointing to the macroblock Can be based on Count the number of coordinate vectors of the occluded area pointing to the macroblock

[0179] According to the m-1 frame After that, you can Passed to the m+xth frame.

[0180] 604. Determine a quantization parameter of a second block corresponding to the first block in the pre-encoded frame according to the first quantity and the second quantity, the quantization parameter of the second block being a target quantization parameter.

[0181] Step 404 may include: determining a change in the quantization parameter of the second block according to the first quantity and the second quantity; and determining the quantization parameter of the second block according to the change in the quantization parameter of the second block and the global quantization parameter.

[0182] The second block is the jth block in the m+xth frame, and the change in the quantization parameter of the second block can be determined by the following relationship:

[0183] Among them, deltaQP j Indicates the change in the quantization parameter of the second block, A is a constant, ε is a constant, is the first quantity, The second quantity.

[0184] If the global quantization parameter is 23, the QP of the jth block in the m+xth frame is j =23+deltaQP j , the QP j That is the target quantization parameter of the jth block in the m+xth frame.

[0185] when Greater than When deltaQP j is a negative number, QP j It will be smaller than the global quantization parameter 23, that is, a smaller QP is used for high motion areas j Encoding; when Less than When deltaQP j is a positive number, QP j It will be greater than the global quantization parameter 23, that is, a larger QP is used for the occluded area j Encoding. When equal When deltaQP j 0, that is, QP j Equal to the global quantization parameter 23.

[0186] 605.Use QP j Encode the jth block in the m+xth frame and compare it with QP j The relevant information is embedded in the code stream of the jth block.

[0187] With QP j The relevant information may be a descriptor, which is used to indicate whether the jth block is a motion block or an occluded block, and when the jth block is a motion block, the target quantization parameter used to encode the jth block is less than the global quantization parameter, and when the jth block is an occluded block, the target quantization parameter used to encode the second block is greater than the global quantization parameter.

[0188] The descriptor can be 0 or 1, or other forms of indication, for example: 0 to indicate that the second block is a motion block, and 1 to indicate that the second block is an occluded block; or 1 to indicate that the second block is a motion block, and 0 to indicate that the second block is an occluded block. This application does not limit this. Indicating the target quantization parameter to be used when encoding the second block using the descriptor can improve the rationality of the allocation of coding resources for the second block.

[0189] Specifically, it can be: The descriptor representing the motion can be embedded into the bitstream of the jth block for subsequent decoding. It means that the jth block is an occluded descriptor, so as to indicate that the jth block is an occluded block, and the descriptor of the occluded block is embedded in the code stream.

[0190] Repeat the above operation for each block in the m+xth frame to complete the encoding of the m+xth frame.

[0191] By repeating the above process for each video frame in the video sequence, the encoding of all frames in the video sequence can be completed.

[0192] It should be noted that the process described in the above embodiment can be applied to the H.264 encoder. A YUV sequence is input to the H.264 encoder. The resolution of the sequence can be 1920x1080, the frame rate is 30fps, the IPPP prediction structure, and the global QP can be set to 23. Through the video encoding method provided above, each frame in the YUV sequence can be encoded.

[0193] If in the hard encoder scenario, and QP j The relevant information can be deltaQP j , deltaQP j Embedded in the code stream of block j for subsequent decoding.

[0194] The above-mentioned video encoding scheme provided by the embodiment of the present application can also be applied to the scenario of secondary compression. The compressed video data can be decoded first, and the decoded YUV sequence can be input into the encoder to execute the video encoding process provided by the embodiment of the present application. For example, the input is an H.264 standard code stream v, and it is decoded into a YUV sequence of 300 frames in total. According to the GOP length of 60 frames, it is divided into 5 sub-video sequences, v1-v5. For v1-v5, the global QP can be set to 23. During the secondary encoding process, the v1 sub-video sequence can be encoded first to obtain the secondary encoded code stream v'1, and then the v2-v5 video sequences can be encoded one by one to obtain the secondary encoded code streams v'2~v'5, which are merged with v'1 to obtain the final secondary compressed code stream v'.

[0195] The above-mentioned encoding scheme provided in the embodiment of the present application can also be applied to the H.265 encoder. The process of applying to the H.265 encoder is the same as the above process, except that the block in the previous frame can be a macroblock (MB) in H.264, and in the H.265 encoder, the block in the previous frame can be a prediction unit (PU). Of course, the compression format of the present application is not limited to H.264 and H.265, and can also be applied to other compression formats, such as: H.266, etc.

[0196] In the coding scheme provided by the embodiment of the present application, the coded frame and the pre-coded frame are two video frames with a certain interval, and the target area prediction in the pre-coded frame can also be realized, thereby improving the diversity of the target area prediction in the pre-coded frame.

[0197] The video encoding method provided in the above embodiment of the present application relies on inter-frame motion vector tracing to adaptively adjust the quantization parameters, and then uses the adaptively adjusted quantization parameters for encoding, and the encoding efficiency of the video sequence containing the moving body is significantly improved. In order to better illustrate that the encoding efficiency of the present application has been greatly improved, the applicant used four video sequences and conducted experiments using the first scheme and the second scheme for each video sequence. The first scheme is to use the adaptive quantization parameters of the present application for encoding, and the second scheme is to use the global quantization parameters for encoding; when the values ​​of the global quantization parameters are 22, 27, 32 and 37 respectively, when the video sequence is a high-motion video sequence, the four video sequences are encoded using the first scheme and the second scheme respectively, and the average speed increase reaches 23.67%. It can be seen that the encoding efficiency is greatly improved when the scheme of the present application is used for encoding.

[0198] The video coding scheme provided in this application significantly improves coding efficiency when encoding high-motion video sequences, and also slightly improves coding efficiency when encoding static or low-motion video sequences. Therefore, the coding efficiency of both high-motion and static or low-motion video sequences can be improved when using the scheme provided in this application for video coding.

[0199] The encoding scheme provided in the above embodiment of the present application can realize adaptive adjustment of the quantization parameters of each frame or each block, regardless of whether the encoded frame and the pre-encoded frame are continuous video frames or non-continuous video frames, thereby improving the allocation efficiency of encoding resources. Moreover, the algorithm complexity of the scheme provided in the embodiment of the present application is low, and a large amount of frame caching and pre-encoding is not required, which not only improves the encoding efficiency, but also reduces the encoding delay and improves the real-time performance of video processing.

[0200] The above describes the video encoding method. Next, in conjunction with FIG. 7 , a video encoding apparatus 70 provided in an embodiment of the present application is described. When encoding a video sequence, the apparatus 70 includes:

[0201] The prediction unit 701 is configured to predict at least one target region in the pre-encoded frame according to motion information of the first moving object in the encoded frame, where the at least one target region is related to the first moving object.

[0202] The encoding unit 702 is configured to encode video data corresponding to at least one target region using a target quantization parameter, where the target quantization parameter is different from a global quantization parameter of the video sequence.

[0203] Optionally, when there is only one target area, the target area is the first area or the second area; when there are multiple target areas, the multiple target areas include the first area and the second area; wherein the first area is the moving area of ​​the first moving body in the pre-encoded frame, and the target quantization parameter corresponding to the first area is less than the global quantization parameter; the second area is the area in the pre-encoded frame, and the second area is occluded by the first moving body in the next frame of the pre-encoded frame, and the target quantization parameter corresponding to the second area is greater than the global quantization parameter.

[0204] Optionally, the encoded frame and the pre-encoded frame are consecutive frames or non-consecutive frames in a video sequence.

[0205] Optionally, the prediction unit 701 is specifically configured to:

[0206] When each video frame of the video sequence includes multiple blocks, position information of the motion block in the pre-coded frame is predicted based on a first-order motion vector of the motion block in the coded frame, an interval between the coded frame and the pre-coded frame, and position information of the motion block in the coded frame; wherein the motion block in the coded frame is a block corresponding to a first moving object in the coded frame, and the motion block in the pre-coded frame is a block corresponding to the predicted first moving object in the pre-coded frame;

[0207] The first region is determined according to position information of the motion block in the pre-encoded frame.

[0208] Optionally, the prediction unit 701 is specifically configured to:

[0209] When each video frame of a video sequence includes multiple blocks, position information of an occluded block in a pre-coded frame is predicted based on a first-order motion vector of a motion block in an encoded frame, an interval between the encoded frame and a pre-coded frame, and position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is a block corresponding to a first moving object in the encoded frame, and the occluded block in the pre-coded frame is a block predicted to be occluded by the first moving object in a frame next to the pre-coded frame;

[0210] The second area is determined according to the position information of the occluded block in the pre-encoded frame.

[0211] Optionally, the prediction unit 701 is specifically configured to:

[0212] Determining a motion vector of a motion block in a pre-coded frame based on a first-order motion vector of the motion block in the coded frame and an interval between the coded frame and the pre-coded frame; wherein the interval between the coded frame and the pre-coded frame is used to indicate a multiple relationship between the motion vector of the motion block in the pre-coded frame and the first-order motion vector;

[0213] The position information of the motion block in the pre-encoded frame is determined according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the pre-encoded frame.

[0214] Optionally, the prediction unit 701 is specifically configured to:

[0215] Determining, based on a first-order motion vector of a motion block in a coded frame and an interval between the coded frame and the pre-coded frame, a motion vector of a motion block in a frame next to the pre-coded frame; wherein the interval between the coded frame and the pre-coded frame is used to indicate a multiple relationship between the motion vector of the motion block in the frame next to the pre-coded frame and the first-order motion vector;

[0216] The position information of the blocked block in the pre-encoded frame is determined according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the next frame of the pre-encoded frame.

[0217] Optionally, the prediction unit 701 is further configured to: when there are multiple target areas;

[0218] determining, based on position information of the motion block in the pre-encoded frame, a first number of coordinate vectors pointing to a motion region of a first block, where the first block is any block in the encoded frame;

[0219] determining, according to position information of the occluded block in the pre-encoded frame, a second number of coordinate vectors pointing to the occluded area of ​​the first block;

[0220] A quantization parameter of a second block corresponding to the first block in the pre-encoded frame is determined according to the first quantity and the second quantity, and the quantization parameter of the second block is a target quantization parameter.

[0221] Optionally, the prediction unit 701 is specifically configured to:

[0222] determining a change amount of a quantization parameter of the second block according to the first amount and the second amount;

[0223] The quantization parameter of the second block is determined according to the change amount of the quantization parameter of the second block and the global quantization parameter.

[0224] Optionally, the prediction unit 701 is specifically configured to:

[0225] Determine the change in the quantization parameter of the second block according to the following relationship:

[0226] Among them, deltaQP j Indicates the change in the quantization parameter of the second block, A is a constant, ε is a constant, is the first quantity, The second quantity.

[0227] Optionally, the prediction unit 701 is further configured to:

[0228] A descriptor is embedded in the second block, where the descriptor is used to indicate that the second block is a motion block or an occluded block. When the second block is a motion block, a target quantization parameter used for encoding the second block is less than a global quantization parameter. When the second block is an occluded block, a target quantization parameter used for encoding the second block is greater than the global quantization parameter.

[0229] Optionally, in a hard encoder scenario, the prediction unit 701 is further configured to: embed deltaQP in the second block j , deltaQP j Used to determine the corresponding target quantization parameter when encoding the second block.

[0230] Optionally, the video sequence is included in the decoded video stream.

[0231] Optionally, the blocks in each video frame are macroblocks MB in an H.264 scenario, or the blocks in each video frame are prediction units PU in an H.265 scenario.

[0232] The functions of the various units in the video encoding device 70 introduced in this application can be understood by referring to the corresponding contents in the method embodiments introduced above, and will not be repeated here.

[0233] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program, which, when executed on a computer, enables the computer to execute the method described in the embodiments shown in the aforementioned Figures 3 to 6.

[0234] An embodiment of the present application also provides a video encoding device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to execute the method shown in any of the embodiments in Figures 3 to 6 above.

[0235] The present application also provides a digital processing chip. The digital processing chip integrates circuitry and one or more interfaces for implementing the functions of the encoder 200, processor 201, or both. When the digital processing chip integrates memory, it can perform the method steps of any one or more of the aforementioned embodiments. When the digital processing chip does not integrate memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the methods of the aforementioned embodiments based on program code stored in the external memory.

[0236] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes the steps in the method described in any of the embodiments in Figures 3 to 6 above.

[0237] An embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the steps of the method described in any one of the embodiments in FIG. 3 to FIG. 6 .

[0238] The video encoding device provided in the embodiments of the present application may be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit, so that the chip in the computer device executes the video encoding method described in the embodiments shown in Figures 3 to 6 above. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0239] Specifically, the aforementioned processing unit or processor may include a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0240] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0241] In addition, an embodiment of the present application also provides a device for storing a code stream, including at least one storage medium and a communication interface; the communication interface is used to receive or send a code stream; at least one storage medium is used to store a code stream; the code stream is encoded by an encoder executing the video encoding method described in the embodiments shown in Figures 3 to 6 above.

[0242] In a tenth aspect, the present application provides a method for storing a code stream, comprising: receiving a code stream through a communication interface; storing the code stream in one or more storage media, wherein the code stream is encoded by an encoder executing the video encoding method described in the embodiments shown in Figures 3 to 6 above.

[0243] In an eleventh aspect, the present application provides a system for distributing a bitstream, comprising at least one storage medium and a video streaming device; the at least one storage medium is used to store the bitstream, where the bitstream is encoded by an encoder executing the video encoding method described in the embodiments shown in FIG. 3 to FIG. 6 ;

[0244] The video stream device is used for sending a target code stream in at least one storage medium to the decoder in response to a request from the decoder.

[0245] A twelfth aspect of the present application provides a method for distributing a bitstream, including: receiving a first request; selecting a target bitstream from at least one storage medium in response to the first request; and sending the target bitstream to a destination device; the at least one storage medium is used to store the bitstream, and the target bitstream is encoded by an encoder executing the video encoding method described in the embodiments shown in Figures 3 to 6 above.

[0246] A thirteenth aspect of the present application provides a system for processing a code stream, including an image source device, an encoder device, one or more storage media, and a destination device;

[0247] The image source device is used to provide image data;

[0248] The encoder device is used to obtain image data from the image source device through the interface and encode the image data to obtain one or more code streams. The code streams are encoded by the encoder device using the video encoding method described in the embodiments shown in Figures 3 to 6 above.

[0249] The encoder device is used to store one or more code streams into one or more storage media; or the encoder device is used to encapsulate one or more code streams to obtain a transmission code stream;

[0250] The encoder device is used to transmit the transport stream to the destination device through a communication link or a communication network; the destination device is used to decapsulate the transport stream to obtain one or more code streams;

[0251] The destination device is used to decode one or more code streams to obtain decoded data.

[0252] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0253] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0254] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a server, or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A method for video encoding, characterized in that, When encoding a video sequence, the method includes: Predicting at least one target region in a pre-coded frame based on the motion information of a first moving object in an encoded frame, where the at least one target region is related to the first moving object; Encoding the video data corresponding to the at least one target region using a target quantization parameter that is different from the global quantization parameter of the video sequence.

2. The method according to claim 1, wherein When there is one target region, the target region is a first region or a second region; when there are multiple target regions, the multiple target regions include the first region and the second region; wherein The first region is the motion region of the first moving object in the pre-coded frame, and the target quantization parameter corresponding to the first region is less than the global quantization parameter; The second region is a region in the pre-coded frame and is occluded by the first moving object in the next frame of the pre-coded frame, and the target quantization parameter corresponding to the second region is greater than the global quantization parameter.

3. The method according to claim 2, wherein The encoded frame and the pre-coded frame are consecutive frames or non-consecutive frames in the video sequence.

4. The method according to claim 2 or 3, characterized in that, Each video frame of the video sequence includes multiple blocks. The predicting at least one target region in the pre-coded frame based on the motion information of a first moving object in the encoded frame includes: Predicting the position information of a motion block in the pre-coded frame based on the first-order motion vector of the motion block in the encoded frame, the interval between the encoded frame and the pre-coded frame, and the position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is the block corresponding to the first moving object in the encoded frame, and the motion block in the pre-coded frame is the predicted block corresponding to the first moving object in the pre-coded frame; Determining the first region according to the position information of the motion block in the pre-coded frame.

5. The method according to claim 2 or 3, characterized in that, Each video frame of the video sequence includes multiple blocks. The predicting at least one target region in the pre-coded frame based on the motion information of a first moving object in the encoded frame includes: Predicting the position information of an occluded block in the pre-coded frame based on the first-order motion vector of the motion block in the encoded frame, the interval between the encoded frame and the pre-coded frame, and the position information of the motion block in the encoded frame; wherein the motion block in the encoded frame is the block corresponding to the first moving object in the encoded frame, and the occluded block in the pre-coded frame is the predicted block that will be occluded by the first moving object in the next frame of the pre-coded frame; Determining the second region according to the position information of the occluded block in the pre-coded frame.

6. The method according to claim 4, characterized in that The predicting the position information of a motion block in the pre-coded frame based on the first-order motion vector of the motion block in the encoded frame, the interval between the encoded frame and the pre-coded frame, and the position information of the motion block in the encoded frame includes: Determine the motion vector of the motion block in the precoded frame according to the first-order motion vector of the motion block in the encoded frame and the interval between the encoded frame and the precoded frame; wherein, the interval between the encoded frame and the precoded frame is used to indicate the multiple relationship between the motion vector of the motion block in the precoded frame and the first-order motion vector; Determine the position information of the motion block in the precoded frame according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the precoded frame.

7. The method according to claim 5, wherein The predicting the position information of the occluded block in the precoded frame according to the first-order motion vector of the motion block in the encoded frame, the interval between the encoded frame and the precoded frame, and the position information of the motion block in the encoded frame includes: Determine the motion vector of the motion block in the next frame of the precoded frame according to the first-order motion vector of the motion block in the encoded frame and the interval between the encoded frame and the precoded frame; wherein, the interval between the encoded frame and the precoded frame is used to indicate the multiple relationship between the motion vector of the motion block in the next frame of the precoded frame and the first-order motion vector; Determine the position information of the occluded block in the precoded frame according to the position information of the motion block in the encoded frame and the motion vector of the motion block in the next frame of the precoded frame.

8. The method according to any one of claims 4-7, characterized in that When there are multiple target regions, the method further includes: Determine the first quantity of the coordinate vectors of the motion regions pointing to the first block according to the position information of the motion blocks in the precoded frame, where the first block is any block in the encoded frame; Determine the second quantity of the coordinate vectors of the occluded regions pointing to the first block according to the position information of the occluded blocks in the precoded frame; Determine the quantization parameter of the second block corresponding to the first block in the precoded frame according to the first quantity and the second quantity, and the quantization parameter of the second block is the target quantization parameter.

9. The method according to claim 8, characterized in that, The determining the quantization parameter of the second block corresponding to the first block in the precoded frame according to the first quantity and the second quantity includes: Determine the change amount of the quantization parameter of the second block according to the first quantity and the second quantity; Determine the quantization parameter of the second block according to the change amount of the quantization parameter of the second block and the global quantization parameter.

10. The method according to claim 9, characterized in that, The determining the change amount of the quantization parameter of the second block according to the first quantity and the second quantity includes: Determine the change amount of the quantization parameter of the second block according to the following relationship; Among them, deltaQP j represents the change amount of the quantization parameter of the second block, A is a constant, and ε is a constant. is the first quantity, Is the second quantity.

11. The method according to any one of claims 8 - 10, characterized in that, The method further includes: Embed a descriptor in the second block, where the descriptor is used to indicate that the second block is a motion block or an occluded block, and when the second block is a motion block, the target quantization parameter used for encoding the second block is less than the global quantization parameter, and when the second block is an occluded block, the target quantization parameter used for encoding the second block is greater than the global quantization parameter.

12. The method according to claim 10, wherein In the scenario of a hard encoder, the method further includes: Embed deltaQP in the second block j , where the deltaQP j is used to determine the corresponding target quantization parameter when encoding the second block.

13. The method according to any one of claims 1 to 12, characterized in that, The video sequence is included in the decoded video stream.

14. The method according to any one of claims 4 - 12, characterized in that The block in each video frame is a macroblock MB in the H.264 scenario, or the block in each video frame is a prediction unit PU in the H.265 scenario.

15. A video encoding device, characterized in that, When encoding a video sequence, the device includes: A prediction unit, configured to predict at least one target region in a pre-encoded frame according to the motion information of a first moving object in an encoded frame, where the at least one target region is related to the first moving object; An encoding unit, configured to encode the video data corresponding to the at least one target region using a target quantization parameter, where the target quantization parameter is different from the global quantization parameter of the video sequence.

16. An electronic device, characterized in that, Including: A communication interface, an encoder, a processor, and a memory, where the communication interface is coupled to the encoder, the processor, and the memory, and the memory is used to store programs or instructions. When the programs or instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium. When the instructions run on an electronic device, the electronic device executes the method according to any one of claims 1 to 14.

18. A computer program product, characterized in that, The computer program product includes computer program code. When the computer program code runs on a computer, the computer executes the method according to any one of claims 1 to 14.

19. A chip system, characterized in that, The chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are configured to receive signals from the memory of the electronic device and send signals to the processors, and the signals include computer instructions stored in the memory; when the processors execute the computer instructions, the electronic device executes the method according to any one of claims 1 to 14.

20. A video processing system, characterized in that, Including: An electronic device, where the electronic device is configured to execute the method according to any one of claims 1 to 14.

21. A device for storing a bitstream, characterized in that, Including at least one storage medium and a communication interface; the communication interface is configured to receive or send a bitstream; the at least one storage medium is configured to store the bitstream; the bitstream is generated by an encoder Encoded according to the encoding method of any one of claims 1 to 14.

22. A method for storing a bitstream, characterized in that, Including: Receiving a bitstream through the communication interface; Storing the bitstream in one or more storage mediums, where the bitstream is generated by an encoder Encoded according to the encoding method of any one of claims 1 to 14.

23. A system for distributing a bitstream, characterized in that, Including at least one storage medium and a video stream device; the at least one storage medium is configured to store a bitstream, where the bitstream is generated by an encoder Encoded according to the encoding method of any one of claims 1 to 14; The video stream device is configured to, in response to a request from a decoder, send a target bitstream in the at least one storage medium to the decoder.

24. A method for distributing a bitstream, characterized in that, Including: Receiving a first request; In response to the first request, selecting a target bitstream from at least one storage medium; sending the target bitstream to a destination device; The at least one storage medium is configured to store a bitstream, and the target bitstream is generated by an encoder according to the encoding method of any one of claims 1 to 14.

25. A system for processing a bitstream, characterized in that, Including an image source device, an encoder device, one or more storage mediums, and a destination device; The image source device is configured to provide image data; The encoder device is used to obtain the image data of the image source device through an interface, and encode the image data to obtain one or more code streams, where the code streams are encoded by the encoder device according to any one of the encoding methods in claims 1 to 14; The encoder device is used to store the one or more code streams into one or more storage media; Alternatively, the encoder device is used to encapsulate the one or more code streams to obtain a transport stream; The encoder device is used to transmit the transport stream to the destination device through a communication link or a communication network; The destination device is used to de-encapsulate the transport stream to obtain the one or more code streams; The destination device is used to decode the one or more code streams to obtain decoded data.

Images