Video transcoding method, device, electronic device and storage medium

Abstract

The present application proposes a video transcoding method, apparatus, electronic device and storage medium, which includes: decoding a source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; determining a type of macroblock in the second video frame based on the watermark information and the decoding information; for each macroblock, determining target coding parameters required for encoding the macroblock based on the macroblock type and a rate-distortion optimization (RDO) algorithm; and performing macroblock coding based on the target coding parameters of each macroblock to generate a transcoded target video stream.

Description

Cross-references to related applications

[0001] This application is proposed based on and claims priority to a Chinese patent application having Chinese patent application number "202110487567.6" and filing date of April 30, 2021, the entire contents of which are hereby incorporated by reference into this application. [Technical field]

[0002] The present application relates to the field of image processing technology, and in particular to a video transcoding method, apparatus, electronic device and storage medium. [Background technology]

[0003] In related art, when adding a watermark to a source video stream and transcoding it, the video needs to be decoded first, then the watermark is added to the video frame, and finally encoded. Such a general-purpose transcoding method requires a large amount of calculation, a long execution time, a large loss of video quality, and mismatched input and output code rates. Summary of the Invention

[0004] The present application aims to solve at least one of the technical problems in the related art to some extent.

[0005] An embodiment of a first aspect of the present application proposes a video transcoding method, the method including: decoding a source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; determining a type of macroblock in the second video frame based on the watermark information and the decoding information; for each macroblock, determining target encoding parameters required for encoding the macroblock based on the macroblock type and a rate-distortion optimization (RDO) algorithm; and performing macroblock encoding based on the target encoding parameters of each macroblock to generate a transcoded target video stream.

[0006] An embodiment of a second aspect of the present application proposes a video transcoding apparatus, comprising: a decoding module for decoding a source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; a macroblock type obtaining module for determining a type of a macroblock in the second video frame based on the watermark information and the decoding information; a parameter obtaining module for determining, for each macroblock, target encoding parameters required for encoding the macroblock based on the macroblock type and a rate-distortion optimization (RDO) algorithm; and an encoding module for performing macroblock encoding based on the target encoding parameters of each macroblock to generate a transcoded target video stream.

[0007] An embodiment of a third aspect of the present application proposes an electronic device comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor is capable of performing the video transcoding method provided by the embodiment of the first aspect of the present application.

[0008] An embodiment of the fourth aspect of the present application proposes a computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to perform the video transcoding method provided by the embodiment of the first aspect of the present application.

[0009] An embodiment of the fifth aspect of the present application is a computer program M The proposed computer program, when executed by a processor, realises a video transcoding method provided by an embodiment of the first aspect of the present application. [Brief description of the drawings]

[0010] [Figure 1] 1 is a grammar diagram of H.264. [Diagram 2] 1 is a flowchart of a video transcoding method according to an embodiment of the present application. [Diagram 3] FIG. 2 is a schematic diagram of macroblock classification proposed by an embodiment of the present application; [Figure 4] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. [Diagram 5] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. [Figure 6] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. [Figure 7] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. [Figure 8] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. [Figure 9] FIG. 1 is a block diagram of a video transcoding device according to an embodiment of the present application. [Figure 10] 1 is a schematic configuration diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments of the present application will be described in detail, and examples of the embodiments are shown in the drawings, in which the same or similar reference numerals throughout indicate the same or similar elements, or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are illustrative and are only for the purpose of explaining the present application, and should not be understood as limitations on the present application.

[0012] H.264 is an advanced compression digital video codec standard and a widely used digital video compression format. There are four main profiles of H.264: baseline profile, main profile, high profile and extended profile. Each type of profile uses different coding tools. For example, the baseline profile only supports Context Adaptive Variable Length Coding (CAVLC) and intra-coded (I) / Predictive-coded (P) frames, while the high profile supports Context-based Adaptive Binary Arithmetic Coding (CABAC) and bidirectionally-predicted (B) frames. The baseline, main and high profiles are often used for live video and on-demand.

[0013] As shown in FIG. 1, the H.264 standard encapsulates video data in Network Abstract Layer Units (NALUs), which can be classified into several types, such as Sequence Parameter Sets (SPSs) or picture parameter sets (PPSs), instantaneous decoding refresh slices (IDR slices), and slices. SPSs / PPSs mainly store key video coding parameters such as profile, resolution, and number of reference frames. IDR slices / slices store actual video compression data. Each slice is composed of many macroblocks, and macroblock information mainly includes prediction mode, blocking scheme, and residual.

[0014] Video coding and decoding supports two types of prediction methods: intraframe (intra) and interframe (inter), macroblocks are mainly divided into three types: I / P / B, and frame types are mainly divided into three types: I / P / B. Therefore, for video streams with B frames, the coding order and display order of video frames are different. For P / B frames, the encoder uses multiple reference frames to perform interframe prediction, and at the same time, a long-term reference frame is used to further improve video quality. H.264 interframe prediction supports weighted prediction, which can reduce the residual of motion compensation by selecting the weight of pixels in the reference block. The baseline / main profile uses 4x4 integer transform, while the high profile supports two types of transform: 4x4 and 8x8. In addition to the above coding tools, before outputting the decoded frame, the encoder / decoder also invokes deblocking filtering to improve video quality.

[0015] The generic transcoding realizes transcoding by adding a watermark to the source stream, and the computational complexity of each module is 86% for the encoder, 11% for the decoder, and 69% for the encoder's rate distortion optimization (RDO) function.

[0016] For ease of understanding, the following explanation of the terminology in FIG. 1 continues: 1. Video Coding Layer (VCL) 2. Network Abstraction Layer (NAL) 3. Slice Layer, Slice Header, Slice Data 4. Macro Block (MB) 5. Macroblock Layer, Type, Prediction, Macroblock Pattern, Quantization Parameter (QP), Residual 6. Intra modes, Reference Frames, Motion vectors

[0017] This application proposes a video transcoding method, device, electronic device and storage medium, which can solve the problems of large calculation volume, long execution time, large video quality loss and inconsistent input and output code rates in common transcoding solutions.

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a video transcoding method, device, electronic device, and storage medium according to embodiments of the present application will be described in conjunction with the drawings.

[0019] 2 is a flowchart of a video transcoding method according to an embodiment of the present application. As shown in FIG. 2, the video transcoding method includes the following steps S201 to S204.

[0020] S201, decode a source video stream to obtain a first video frame and decoding information, and insert watermark information into the first video frame to generate a second video frame.

[0021] In an embodiment of the present application, a source video stream can be obtained, the source video stream can be input to a decoder, and the source video stream can be decoded by the decoder to obtain a first video frame in the source video stream and extract decoding information of the first video frame.

[0022] In some embodiments, the decoding information includes slice header information, reference frame reordering information, reference image sequence marks, decoder reference frame lists, reconstructed pixels and macroblock information before deblocking filtering, etc. In some embodiments, the macroblock information includes prediction modes, blocking schemes, residuals, and quantization parameters, etc.

[0023] Insert watermark information into the first video frame to generate a second video frame. On the one hand, the watermark information can play the role of privacy protection, copyright protection or subtitle interpretation for the video frame, and on the other hand, the watermark information also identifies modified pixels in the video frame and instructs the transcoding device to pay attention to the macroblocks containing the modified pixels. In some embodiments, the watermark information can be copyright information, privacy information, subtitle information, etc.

[0024] S202, determining a type of a macroblock in the second video frame according to the watermark information and the decoding information.

[0025] A video frame can be divided into macroblocks, which are the basic unit of coding, and in some embodiments, the size of a macroblock is 16x16.

[0026] In the embodiment of the present application, the watermark information can partially or completely change one or more pixels in some macroblocks. The watermark information area is an area in which pixels in a video frame are modified, and the position of the modified pixel in a second video frame can be determined according to the coordinates of the watermark information, and the macroblock in which the modified pixel in the second video frame is included can be determined as a first type macroblock. That is, the first type indicates that the macroblock is directly related to the watermark information. For non-first type macroblocks, the dependency with the first type macroblock is confirmed according to the prediction mode in the macroblock decoding information, and the type of the macroblock is confirmed according to whether the dependency with the first type macroblock exists. In the embodiment of the present application, the type of any one macroblock whose decoding information depends on the first type macroblock is identified as the second type, and the type of any one macroblock whose decoding information does not depend on the first type macroblock is identified as the third type, that is, when performing intraframe prediction or interframe prediction, the decoding information of the second type macroblock depends on the first type macroblock, and the decoding information of the third type macroblock does not depend on the first type macroblock. Thus, the types of all macroblocks in the second video frame are obtained. The classification of macroblocks is shown in FIG. 3, where logo represents the first type macroblock, quality represents the second type macroblock, and copy represents the third type macroblock.

[0027] The purpose of determining the type of a macroblock is to facilitate selectively obtaining target coding parameters that match the type of the macroblock for that type of macroblock, thereby reducing the amount of calculation and improving the transcoding speed.

[0028] S203, for each macroblock, determine target coding parameters required for coding the macroblock based on the type of the macroblock and a rate-distortion optimization (RDO) algorithm.

[0029] To achieve fast transcoding of macroblocks, in embodiments of the present application, different target coding parameters may correspond to different macroblock types. In some embodiments, the target coding parameters may include prediction modes, blocking methods, motion vectors, integer transform sizes, quantization parameters, and residuals.

[0030] The reference coding pixels of a macroblock are taken as the input of the RDO algorithm, and the reference coding pixels selected for a macroblock often affect the RDO cost value corresponding to the macroblock. In the embodiment of the present application, different reference coding pixels are selected for different types of macroblocks, and after the reference coding pixels are determined, the candidate coding parameters of the macroblock can be further obtained, where the candidate coding parameters can include decoding information. The reference coding pixels and the decoding information of the macroblock are processed by the RDO algorithm, that is, the reference coding pixels and the decoding information of the macroblock are input into the RDO algorithm, and the candidate coding parameters and the corresponding RDO cost values ​​can be obtained. Furthermore, the candidate coding parameters with the smallest RDO cost values ​​can be selected as the target coding parameters required for encoding the macroblock. In the embodiment of the present application, during encoding of a macroblock, instead of directly calling the native RDO algorithm of the encoder, the logic of the RDO algorithm is modified to calculate the RDO cost values ​​based on the candidate coding parameters including the decoding information, and the smallest one is selected as the target coding parameters, thereby reducing the amount of calculation during encoding.

[0031] S204, performing macroblock encoding according to the target encoding parameters of each macroblock to generate a transcoded target video stream.

[0032] For any one of the macroblocks, encode the macroblock according to the H.264 video standard based on the target encoding parameters of the macroblock to generate target encoding data corresponding to a second video frame, and generate a target video stream based on the target encoding data of each frame.

[0033] The video transcoding method proposed by the embodiments of the present application includes: decoding a source video stream to obtain a first video frame and decoding information, inserting watermark information into the first video frame to generate a second video frame, determining a type of macroblock in the second video frame based on the watermark information and the decoding information, determining, for each macroblock, target encoding parameters required for encoding the macroblock based on the macroblock type and a rate-distortion optimization (RDO) algorithm, and performing macroblock encoding based on the target encoding parameters of each macroblock to generate a transcoded target video stream.

[0034] In this method, the coding can be done based on the decoded information during the coding process, which not only reduces the amount of calculation, but also ensures that the frame type of the added watermark information is completely consistent with the source video stream, and there is almost no damage to the image quality, resulting in high video quality.

[0035] 4 is a flowchart of a video transcoding method according to another embodiment of the present application. As shown in FIG. 4, based on the above embodiment, step S203 may include steps S401 to S403.

[0036] S401, determining reference coding pixels of a macroblock based on the type of the macroblock.

[0037] For video transcoding that does not need to insert watermark information, the encoder simply uses the decoding information to re-encode, and the pixels before deblocking filtering at the encoding side and the decoding side are the same. In the embodiment of this application, watermark information needs to be inserted into the source video stream during video transcoding, and the encoder needs to use the decoding information as much as possible to achieve the purpose of efficient and high-quality transcoding.

[0038] For the area modified by watermark information, in order to ensure the accuracy of video transcoding, its reference coding pixels need to directly use the pixels modified by watermark information. Because the first type macroblock contains the area modified by watermark information, while the non-first type macroblock does not contain the area modified by watermark information, there is a difference between the reference coding pixels of the two, and it is necessary to select the appropriate reference coding pixels according to the type of macroblock.

[0039] S402, performing an RDO algorithm process on the decoded information of the macroblock and the reference coded pixels to generate at least one set of candidate coding parameters of the macroblock and an RDO cost value corresponding to each set of candidate coding parameters.

[0040] In some implementations, the encoder uses the macroblock decoded information and the reference coded pixels to generate the coding parameters and the corresponding RDO cost values, and in some implementations, the encoder uses the macroblock decoded information other than the macroblock residual and the reference coded pixels to generate the coding parameters and the corresponding RDO cost values.

[0041] Combining practical application scenarios, the complexity of obtaining coding parameters in native RDO algorithm is high, the amount of calculation is large, and the transcoding speed is low, so that in order to balance the quality and speed of video transcoding, it is necessary to selectively select target coding parameters for macroblocks. In some embodiments, in the embodiments of the present application, the coding parameters generated using macroblock decoding information are taken as one set of candidate coding parameters, and their RDO cost values ​​are obtained, and the coding parameters generated using macroblock decoding information other than macroblock residual are taken as another set of candidate coding parameters, and their RDO cost values ​​are obtained, and a target coding parameter is selected from the candidate coding parameters.

[0042] To simplify calculation, in an embodiment of the present application, the RDO cost values ​​are compared to identify the target coding parameters from the candidate coding parameters, and the RDO cost value can indicate the optimal candidate coding parameters as the target coding parameters for video transcoding.

[0043] S403, determining a target encoding parameter required for encoding the macroblock from at least one set of candidate encoding parameters based on the RDO cost value.

[0044] Among the candidate coding parameters, the candidate coding parameter corresponding to the smallest RDO cost value is selected as the target coding parameter for the macroblock.

[0045] In some embodiments, in order to reduce errors, the embodiments of the present application may preset a threshold value, and when the minimum RDO cost value is greater than the preset threshold value, the native RDO algorithm of the encoder is invoked to obtain the native RDO cost value and native encoding parameters of the macroblock, and the minimum RDO cost value is subsequently compared with the native RDO cost value, and when the minimum RDO cost value is less than or equal to the native RDO cost value, the set of candidate encoding parameters corresponding to the minimum RDO cost value is the target encoding parameters of the macroblock, and when the minimum RDO cost value is greater than the native RDO cost value, the native encoding parameters are the target encoding parameters of the macroblock.

[0046] 5 is a flowchart of a video transcoding method according to another embodiment of the present application. As shown in FIG. 5, based on the above embodiment, when the type of the macroblock is a first type, the encoding process of the macroblock includes the following steps S501 to S503:

[0047] S501, performing type recognition on a macroblock in a second video frame to determine that the type of the macroblock is a first type.

[0048] Referring to step S102, the macroblock in which the modified pixel in the second video frame is included is determined as a first type macroblock, that is, the first type indicates that the macroblock is directly related to the watermark information.

[0049] S502, determining a first region in which pixels in a first type macroblock are not modified with watermark information and a second region in which pixels are modified with watermark information.

[0050] In the implementation, a watermark is inserted into a video frame, and the watermark may only occupy one or more pixels of some macroblocks, i.e., the first type macroblock may include regions modified by watermark information and may include regions not modified by watermark information, so it is necessary to segment the first type macroblock to check its reference coding pixels. In the embodiment of the present application, the region whose pixels are not modified by watermark information is the first region, and the region whose pixels are modified by watermark information is the second region. Here, the reference coding pixel of the second region is the pixel currently input by the encoder, and the currently input pixel is the pixel modified by watermark information.

[0051] For the first region that is not modified by the watermark information, the reconstructed pixels before the deblocking filtering of the decoder can be used as the reference encoding pixels of the first region, which not only improves the encoding quality but also ensures the efficiency of video transcoding since the decoding information is transcoded.

[0052] For the second region modified with the watermark information, its reference coding pixels directly use the pixels modified with the watermark information, thereby ensuring the accuracy of the video transcoding.

[0053] S503, calling a native RDO algorithm of the encoder, obtaining native coding parameters of the first type macroblock according to the reference coding pixels of the first type macroblock, and determining the native coding parameters as target coding parameters of the first type macroblock.

[0054] For the first type macroblock, according to the reference coding pixels obtained in step S502, invoke a native RDO algorithm of the encoder to obtain native coding parameters of the macroblock, and determine the native coding parameters as target coding parameters of the first type macroblock, and perform macroblock coding according to the target coding parameters to generate target coding data corresponding to the second video frame, and further generate a transcoded target video stream.

[0055] The video transcoding method according to the embodiment of the present application not only reduces the amount of calculation, but also ensures that the frame type of the added watermark information is completely consistent with the source video stream, and there is little loss in image quality, resulting in high video quality.

[0056] 6 is a flowchart of a video transcoding method according to another embodiment of the present application. As shown in FIG. 6, based on the above embodiment, when the type of the macroblock is the second type, the encoding process of the macroblock includes the following steps S601 to S608:

[0057] S601, performing type recognition on a macroblock in a second video frame to determine that the type of the macroblock is a second type, and taking the reconstructed pixels before deblocking filtering of the decoder as reference coding pixels.

[0058] Here, the second type indicates that a dependency exists between the macroblock and the macroblock that carries the watermark information.

[0059] Referring to step S102, for a non-first type that does not include modified pixels in the second video frame, determine a type in which macroblock decoding information in a prediction mode depends on a macroblock of the first type as a second type.

[0060] The reconstructed pixels before deblocking filtering in the decoder are non-first type reference coded pixels, ie, the reference coded pixels of the second type macroblock are reconstructed pixels before deblocking filtering.

[0061] S602, generating at least one set of candidate coding parameters for the second type macroblock and its RDO cost value based on the decoding information of the second type macroblock and the reference coding pixels.

[0062] In an embodiment of the present application, the coding parameters generated using the macroblock decoding information and the reference coding pixels are taken as one set of candidate coding parameters, and their RDO cost values ​​are obtained; and the coding parameters generated using the macroblock decoding information other than the macroblock residual and the reference coding pixels are taken as another set of candidate coding parameters, and their RDO cost values ​​are obtained.

[0063] The native RDO algorithm of the encoder has high complexity, and in order to reduce the number of calls of the RDO algorithm, in the embodiment of the present application, the RDO cost values ​​corresponding to each set of candidate encoding parameters are compared to obtain the target encoding parameters.

[0064] S603, determine a minimum RDO cost value, and determine whether the minimum RDO cost value is greater than a preset threshold value.

[0065] RDO is an algorithm module that can compare the merits of video encoders according to the rate distortion (RD) curve, and selects an optimal point on the RD two-dimensional graph, which corresponds to a cost value. The cost value is a value for selecting an optimal encoding method, and the smallest cost value indicates that the encoding parameter corresponding to this cost value is the optimal encoding parameter for the macroblock. Therefore, to select the optimal encoding parameter for the macroblock, the embodiment of the present application first compares the minimum RDO cost value with a preset threshold value to determine whether the minimum RDO cost value is greater than the preset threshold value.

[0066] S604, in response to the minimum RDO cost value being less than or equal to a preset threshold, select a set of candidate encoding parameters corresponding to the minimum RDO cost value as target encoding parameters for a second type macroblock.

[0067] If the minimum RDO cost value is less than or equal to a preset threshold, it indicates that the candidate encoding parameters corresponding to the current minimum RDO cost value meet the requirement, and the set of candidate encoding parameters corresponding to the minimum RDO cost value can be set as the target encoding parameters when encoding the second type macroblock.

[0068] S605, in response to the smallest RDO cost value being greater than the preset threshold, invoke a native RDO algorithm of the encoder to obtain native RDO cost values ​​and native coding parameters of the second type macroblock.

[0069] If the minimum RDO cost value is greater than the preset threshold, it indicates that the candidate coding parameters corresponding to the current minimum RDO cost value may not be the optimal coding parameters, and the native RDO algorithm of the encoder should be invoked to obtain the native RDO cost value and the native coding parameters of the macroblock, and continue to compare the native RDO cost value with the current minimum RDO cost value to update the minimum RDO cost value.

[0070] S606, it is determined whether the minimum RDO cost value is greater than the native RDO cost value.

[0071] S607, if the minimum RDO cost value is less than or equal to the native RDO cost value, the set of candidate coding parameters corresponding to the minimum RDO cost value is set as the target coding parameters of the macroblock.

[0072] If the minimum RDO cost value is less than or equal to the native RDO cost value, a set of candidate encoding parameters corresponding to the minimum RDO cost value is taken as target encoding parameters for the macroblock, and macroblock encoding is performed based on the target encoding parameters to generate target encoding data corresponding to the second video frame, and further to generate a transcoded target video stream.

[0073] S608, if the minimum RDO cost value is greater than the native RDO cost value, the native coding parameters are set as the target coding parameters of the macroblock.

[0074] If the minimum RDO cost value is greater than the native RDO cost value, the native encoding parameters are taken as the target encoding parameters of the macroblock, and macroblock encoding is performed based on the target encoding parameters to generate target encoding data corresponding to the second video frame, and further generate a transcoded target video stream.

[0075] The video transcoding method according to the embodiment of the present application not only reduces the amount of calculation, but also ensures that the frame type of the added watermark information is completely consistent with the source video stream, and there is little loss in image quality, resulting in high video quality.

[0076] 7 is a flowchart of a video transcoding method according to another embodiment of the present application. As shown in FIG. 7, based on the above embodiment, when the type of the macroblock is the third type, the encoding process of the macroblock includes the following steps S701 to S703:

[0077] S701, performing type recognition on a macroblock in a second video frame to determine that the macroblock is a third type macroblock, and taking the reconstructed pixel before deblocking filtering of the decoder as the reference coding pixel.

[0078] Here, the third type of macroblock indicates that there is no dependency between the macroblock and the macroblock that carries the watermark information.

[0079] Referring to step S102, for a non-first type that does not include a modified pixel in the second video frame, determine a third type as a type in which the macroblock decoding information in the prediction mode does not depend on the macroblock of the first type.

[0080] The reconstructed pixels before deblocking filtering in the decoder are non-first type reference coded pixels, ie, the reference coded pixels of the third type macroblock are reconstructed pixels before deblocking filtering.

[0081] S702, generating at least one set of candidate coding parameters for the third type macroblock and its RDO cost value based on the decoding information of the third type macroblock and the reference coding pixels.

[0082] For the specific description of steps S701 and S702, reference may be made to the relevant content description of the above embodiment, and the description will be omitted here.

[0083] S703, select a set of candidate coding parameters corresponding to the smallest RDO cost value as target coding parameters for a third type macroblock.

[0084] To obtain the optimal encoding parameters, the candidate encoding parameters having the smallest RDO cost value can be selected as target encoding parameters required for encoding the macroblocks of the third type macroblocks, and macroblock encoding is performed based on the target encoding parameters to generate target encoded data corresponding to the second video frame, and further generate a transcoded target video stream.

[0085] The video transcoding method according to the embodiment of the present application not only reduces the amount of calculation, but also ensures that the frame type of the added watermark information is completely consistent with the source video stream, and there is little loss in image quality, resulting in high video quality.

[0086] 8 is a flowchart of a video transcoding method according to another embodiment of the present application. As shown in FIG. 8, based on the above embodiment, the video transcoding method further includes steps S801-S802.

[0087] S801, in response to a size mismatch between the encoded data of the target video and the encoded data of the source video, determine a priority for adjusting the quantization value of the macroblock based on the type of the macroblock.

[0088] In response to the size mismatch between the encoded data of the target video and the encoded data of the source video, the quantization value needs to be adjusted to unify the size of the video frames and ensure the stability of the transcoding.

[0089] In the application scenario of inserting watermark information into the source video stream to perform video transcoding, the proportion of the first type macroblocks and the second type macroblocks is small, and the majority of the macroblocks are the third type macroblocks, and the transcoding of the third type macroblocks depends on the decoding information, and the size of the video encoding data thereof is almost unchanged. Therefore, the priority for adjusting the quantization value of the macroblocks is the first type macroblocks, the second type macroblocks, and the third type macroblocks. That is, in response to the size mismatch between the encoding data of the target video and the encoding data of the source video, the quantization value of the first type macroblocks is adjusted first.

[0090] S802, sequentially adjust the quantization value of each type of macroblock according to the priority order, and quantize the macroblock according to the adjusted quantization value.

[0091] In response to a size mismatch between the encoded data of the target video and the encoded data of the source video, first adjust the quantization value of the first type macroblock and quantize the macroblock based on the adjusted quantization value, if the adjusted encoded data of the target video matches the encoded data size of the source video, the quantization adjustment ends; otherwise, adjust the quantization value of the second type macroblock and quantize the macroblock based on the adjusted quantization value for a second time, and continue to determine whether the adjusted encoded data of the target video matches the encoded data size of the source video, if so, the quantization adjustment ends; otherwise, continue to adjust the quantization value of the third type macroblock and quantize the macroblock based on the adjusted quantization value for a third time until the encoded data of the target video matches the encoded data size of the source video.

[0092] Quantization is a lossy compression technique that affects the quality of a video frame, so to guarantee the quality of the transcoded video frame, every adjustment of the quantization value must meet a minimum quality requirement.

[0093] The video transcoding method according to the embodiment of the present application not only reduces the amount of calculation, but also ensures that the frame type of the added watermark information is completely consistent with the source video stream, and there is little loss in image quality, resulting in high video quality.

[0094] As shown in FIG. 9, an embodiment of the present application further provides a video transcoding apparatus 900, which includes: a decoding module 910 for decoding a source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; a macroblock type obtaining module 920 for determining a type of a macroblock in the second video frame based on the watermark information and the decoding information; a parameter obtaining module 930 for determining, for each macroblock, target encoding parameters required for encoding the macroblock based on the macroblock type and a rate-distortion optimization (RDO) algorithm; and an encoding module 940 for performing macroblock encoding based on the target encoding parameters of each macroblock to generate a transcoded target video stream.

[0095] Further, in one possible implementation of the embodiment of the present application, the parameter acquisition module 930 includes: The coding apparatus further includes an encoded pixel acquisition module 931 for determining reference encoded pixels of the macroblock based on the type of the macroblock, a preliminary parameter acquisition module 932 for performing RDO algorithm processing on the decoded information and reference encoded pixels of the macroblock to generate at least one set of candidate encoding parameters of the macroblock and an RDO cost value corresponding to each set of candidate encoding parameters, and a target parameter acquisition module 933 for determining target encoding parameters required for encoding the macroblock from the at least one set of candidate encoding parameters based on the RDO cost value.

[0096] Furthermore, in one possible implementation form of an embodiment of the present application, the coded pixel acquisition module 931 further indicates that for non-first type macroblocks, the reconstructed pixels before deblocking filtering of the decoder are reference coded pixels, and for the first type macroblocks, a first region in which pixels in the first type macroblock are not modified with watermark information and a second region in which pixels are modified, where the reference coded pixels of the first region are reconstructed pixels before deblocking filtering of the decoder, the reference coded pixels of the second region are pixels currently input by the encoder, and the currently input pixels are pixels modified with watermark information, where the first type macroblock is associated with watermark information.

[0097] Furthermore, in one possible implementation form of an embodiment of the present application, the parameter acquisition module 930 further selects a set of candidate encoding parameters corresponding to the smallest RDO cost value as target encoding parameters for the macroblock in response to the type of the macroblock being a non-first type.

[0098] Furthermore, in one possible implementation form of an embodiment of the present application, the target parameter acquisition module 933 further invokes a native RDO algorithm of the encoder to obtain a native RDO cost value and native encoding parameters of the macroblock in response to the type of the macroblock being a second type and the minimum RDO cost value being greater than a predetermined threshold, where the second type indicates that a dependency exists between the macroblock and a macroblock carrying watermark information, and if the minimum RDO cost value is less than or equal to the native RDO cost value, a set of candidate encoding parameters corresponding to the minimum RDO cost value is the target encoding parameters of the macroblock, and if the minimum RDO cost value is greater than the native RDO cost value, the native encoding parameters are the target encoding parameters of the macroblock.

[0099] Furthermore, in one possible implementation form of an embodiment of the present application, the target parameter acquisition module 933 further, in response to the type of the macroblock being a first type, invokes a native RDO algorithm of the encoder to generate native encoding parameters for the macroblock, and determines the native encoding parameters as target encoding parameters for the macroblock.

[0100] Further, in one possible implementation of an embodiment of the present application, the encoding module 940 further performs macroblock encoding on the macroblock based on the target encoding parameters of the macroblock to generate target encoding data corresponding to a second video frame, and generates a target video stream based on the target encoding data of each frame.

[0101] Furthermore, in one possible implementation form of the embodiment of the present application, the macroblock type acquisition module 920 further acquires a modified pixel when watermark information is inserted into the second video frame, and determines the type of the macroblock containing the modified pixel as a first type, and for any one of the remaining macroblocks, determines whether any one of the macroblocks depends on a macroblock of the first type based on the decoding information of any one of the macroblocks, and determines the type of any one of the macroblocks that depends on the macroblock of the first type as a second type, and determines the type of any one of the macroblocks that does not depend on the macroblock of the first type as a third type.

[0102] Further, in one possible implementation of an embodiment of the present application, the encoding module 940 further determines a priority for adjusting the quantization values ​​of the macroblocks based on the type of the macroblock in response to a mismatch in size between the encoded data of the target video and the encoded data of the source video, sequentially adjusts the quantization values ​​of each type of macroblock according to the priority, and quantizes the macroblocks based on the adjusted quantization values.

[0103] An embodiment of the present application further provides an electronic device.

[0104] 10 is a schematic diagram of an electronic device provided by an embodiment of the present application. As shown in FIG. 10, the electronic device 100 includes a storage medium 110, a processor 120, and a computer program that is stored in the memory 110 and is executable by the processor 120. M When the processor executes the computer program, the above-mentioned video transcoding method is realized.

[0105] Those skilled in the art should appreciate that the embodiments of the present invention may be provided as a method, a system, or a computer program product. Thus, the present application may employ an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may employ a computer program product embodied in one or more computer usable storage media (including, but not limited to, magnetic disk memory, CD-ROM, optical memory, etc.) that contains computer usable program code. M The form can be adopted.

[0106] The present application relates to a method, a device (system) and a computer program according to the embodiments of the present application. M The description has been given with reference to flowcharts and / or block diagrams. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing apparatus to generate an apparatus in which the instructions executed by the processor of the computer or other programmable data processing apparatus are used to realize the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0107] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner to produce an article of manufacture including an instruction apparatus that implements the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams, the instructions stored in the computer readable memory.

[0108] These computer program instructions may also be loaded into a computer or other programmable data processing apparatus to execute a series of operational steps on the computer or other programmable apparatus to generate a computer-implemented process, whereby the instructions executing on the computer or other programmable apparatus provide steps for realizing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the block diagrams.

[0109] An embodiment of the present application further provides a computer-readable storage medium having computer instructions stored thereon, which causes a computer to perform the video transcoding method of the above embodiment.

[0110] The embodiments of the present application are M It is further provided that, when the computer program is executed by a processor, the video transcoding method of the above embodiment is realized.

[0111] It should be noted that in the claims, any reference signs between parentheses shall not be construed as limiting the claim. The word "comprise" does not exclude parts or steps not mentioned in the claim. The word "a" or "one" preceding a part does not exclude the presence of a plurality of such parts. The application can be implemented by means of hardware comprising several different components, and by a computer with appropriate programming. In claims enumerating several units of apparatus, several of these apparatuses may be embodied by one and the same hardware. The use of words such as first, second, and third does not indicate any order. These words can be interpreted as names.

[0112] Additionally, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying a relative importance and number of technical features depicted. Thus, a feature qualified as "first" or "second" may explicitly or implicitly include one or more of said features. In the description of this application, "plurality" means two or more, unless otherwise specifically and explicitly limited.

[0113] All embodiments of the present disclosure may be implemented alone or in combination with other embodiments, all of which are considered within the scope of protection of the claims of the present disclosure.

Claims

1. decoding a source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; determining a type of a macroblock in the second video frame based on the watermark information and the decoding information; For each macroblock, determining reference coding pixels for said macroblock based on the type of said macroblock; performing an RDO algorithm process on the decoded information of the macroblock and the reference coded pixels to generate at least one set of candidate coding parameters for the macroblock and an RDO cost value corresponding to each set of candidate coding parameters; determining, based on said RDO cost value, target coding parameters required for coding said macroblock from said at least one set of candidate coding parameters; performing macroblock encoding based on the target encoding parameters for each macroblock to generate a transcoded target video stream; Including, determining reference coding pixels for the macroblock based on the type of the macroblock, for non-first type macroblocks, the reference coding pixels are reconstructed pixels before deblocking filtering in a decoder; determining, for a first type macroblock, a first region in which pixels in the first type macroblock are not modified with the watermark information and a second region in which pixels are modified, wherein reference coding pixels of the first region are reconstructed pixels before deblocking filtering of the decoder, and reference coding pixels of the second region are pixels currently input by an encoder, and the currently input pixels are pixels modified with the watermark information; Including, indicating that the first type of macroblock is associated with the watermark information.

2. determining, based on the RDO cost value, from the at least one set of candidate coding parameters, a target coding parameter required for coding the macroblock, 2. The method of claim 1, further comprising the step of: in response to the type of the macroblock being a non-first type, selecting a set of candidate encoding parameters corresponding to a minimum of the RDO cost value as target encoding parameters for the macroblock.

3. in response to the type of the macroblock being a second type and the minimum RDO cost value being greater than a preset threshold, invoking a native RDO algorithm of the encoder to obtain a native RDO cost value and native coding parameters for the macroblock, the second type indicating that a dependency exists between the macroblock and a macroblock carrying the watermark information; if the minimum RDO cost value is less than or equal to the native RDO cost value, determining a set of candidate coding parameters corresponding to the minimum RDO cost value as target coding parameters for the macroblock; 2. The method of claim 1, further comprising the step of: if the minimum of the RDO cost values ​​is greater than the native RDO cost value, setting the native coding parameters as the target coding parameters for the macroblock.

4. 3. The method of claim 2, further comprising, in response to the type of the macroblock being the first type, invoking a native RDO algorithm of the encoder to generate native coding parameters for the macroblock and determining the native coding parameters as target coding parameters for the macroblock.

5. performing macroblock encoding based on the target encoding parameters for each of the macroblocks to generate a transcoded target video stream; 2. The method of claim 1, further comprising the steps of: performing macroblock encoding on the macroblock based on target encoding parameters of the macroblock to generate target encoding data corresponding to the second video frame; and generating the target video stream based on the target encoding data for each frame.

6. determining a type of a macroblock in the second video frame based on the watermark information and the decoding information, obtaining a modified pixel when the watermark information is inserted into the second video frame, and determining a type of a macroblock including the modified pixel as a first type; for any one of the remaining macroblocks, determining whether the any one of the macroblocks depends on the first type macroblock based on the decoding information of the any one of the macroblocks; determining a type of the one of the macroblocks that is dependent on the one of the macroblocks of the first type as a second type, and determining a type of the one of the macroblocks that is not dependent on the one of the macroblocks of the first type as a third type; The method of claim 1 , comprising:

7. prior to performing macroblock encoding based on the target encoding parameters for each of the macroblocks, determining a priority for adjusting quantization values ​​of the macroblocks based on a type of the macroblocks in response to a mismatch in size between the encoded data of the target video stream and the encoded data of the source video stream; adjusting the quantization value of each type of macroblock in sequence according to the priority order, and quantizing the macroblock based on the adjusted quantization value; The method of claim 1 , comprising:

8. a decoding module for decoding the source video stream to obtain a first video frame and decoding information, and inserting watermark information into the first video frame to generate a second video frame; a macroblock type obtaining module for determining a type of a macroblock in the second video frame based on the watermark information and the decoding information; a parameter obtaining module for determining, for each macroblock, target coding parameters required for coding of said macroblock based on the type of said macroblock and a rate-distortion optimization (RDO) algorithm; an encoding module for performing macroblock encoding based on the target encoding parameters for each macroblock to generate a transcoded target video stream; Equipped with The parameter acquisition module: a coding pixel obtaining module for determining reference coding pixels of the macroblock based on a type of the macroblock; a preliminary parameter acquisition module for performing an RDO algorithm process on the decoded information of the macroblock and the reference coded pixels to generate at least one set of candidate coding parameters for the macroblock and an RDO cost value corresponding to each set of candidate coding parameters; a target parameter obtaining module for determining, based on the RDO cost value, target encoding parameters required for encoding the macroblock from the at least one set of candidate encoding parameters; Equipped with The encoded pixel acquisition module: For non-first type macroblocks, the reconstructed pixels before deblocking filtering in the decoder are the reference coded pixels; For a first type macroblock, determining a first region in which pixels in the first type macroblock are not modified with the watermark information and a second region in which pixels are modified, the reference coding pixels of the first region being reconstructed pixels before deblocking filtering of the decoder, the reference coding pixels of the second region being pixels currently input by an encoder, and the currently input pixels being pixels modified with the watermark information; An apparatus for transcoding video, the apparatus indicating that the first type of macroblock is associated with the watermark information.

9. The parameter acquisition module:

9. The video transcoding apparatus of claim 8, wherein in response to the type of the macroblock being non-first type, a set of candidate encoding parameters corresponding to the smallest RDO cost value is selected as target encoding parameters for the macroblock.

10. The target parameter acquisition module: In response to the type of the macroblock being a second type and the minimum RDO cost value being greater than a preset threshold, calling a native RDO algorithm of the encoder to obtain a native RDO cost value and native coding parameters for the macroblock, the second type indicating that a dependency exists between the macroblock and a macroblock carrying the watermark information; if the minimum RDO cost value is less than or equal to the native RDO cost value, a set of candidate coding parameters corresponding to the minimum RDO cost value is set as target coding parameters for the macroblock; 9. The apparatus of claim 8, wherein if the minimum RDO cost value is greater than the native RDO cost value, the native coding parameters are taken as the target coding parameters for the macroblock.

11. The target parameter acquisition module:

10. The video transcoding apparatus of claim 9, further comprising: in response to the type of the macroblock being the first type, invoking a native RDO algorithm of the encoder to generate native coding parameters for the macroblock, and determining the native coding parameters as target coding parameters for the macroblock.

12. The encoding module:

9. The video transcoding device of claim 8, further comprising: performing macroblock encoding on the macroblock based on a target encoding parameter of the macroblock to generate target encoding data corresponding to the second video frame; and generating the target video stream based on the target encoding data of each frame.

13. The macroblock type acquisition module includes: Obtaining a modified pixel when the watermark information is inserted into the second video frame, and determining a type of a macroblock including the modified pixel as a first type; for any one of the remaining macroblocks, determining whether the any one of the remaining macroblocks depends on the first type macroblock based on the decoding information of the any one of the remaining macroblocks; 9. The video transcoding device of claim 8, further comprising: determining a type of any one of the macroblocks that depends on the first type of macroblock as a second type; and determining a type of any one of the macroblocks that does not depend on the first type of macroblock as a third type.

14. The encoding module: determining a priority for adjusting quantization values ​​of the macroblocks based on a type of the macroblocks in response to a mismatch in size between the encoded data of the target video stream and the encoded data of the source video stream; The video transcoding apparatus of claim 8 , further comprising: adjusting the quantization value of each type of macroblock in sequence according to the priority order; and quantizing the macroblock based on the adjusted quantization value.

15. At least one processor; a memory communicatively coupled to the at least one processor; Equipped with 8. An electronic device, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor performs the method of any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: A non-transitory computer readable storage medium having computer instructions that cause a computer to perform the method of any one of claims 1 to 7.

17. A computer program product which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images