A transcoding method and system of a video, an electronic device and a storage medium

Abstract

This application provides a video transcoding method, system, electronic device, and storage medium, belonging to the field of video processing technology. The method includes: acquiring a video stream; decapsulating the video stream to obtain decapsulated data; extracting dynamic metadata from the video stream; traversing the dynamic metadata to determine if a scene change exists; when traversing the dynamic metadata determines that a scene change exists, recording the target dynamic metadata corresponding to the scene change; determining the target frame number based on the target dynamic metadata; determining the scene-changing frame and non-scene-changing frames in the video stream based on the target frame number; adjusting the encoding parameters of the scene-changing frames and / or non-scene-changing frames during re-encoding; and performing video encoding processing based on the adjusted encoding parameters to obtain a transcoded bitstream. In this way, dynamic metadata is used to identify video scene changes, and re-encoding is performed based on scene changes to adjust the bitrate. Non-scene-changing frames reduce the number of code bytes, saving total codewords and improving encoding performance.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video transcoding method, system, electronic device, and storage medium. Background Technology

[0002] With the booming development of the high-definition video industry, users' demands for video picture quality are increasing. As the ultra-high-definition ecosystem rapidly evolves, video playback platforms are diversifying, requiring the same content to be adapted to the playback needs of multiple platforms. Given the significant differences in video formats supported by various platforms, the market demand for professional video transcoding tools continues to grow. Against this backdrop, how to achieve efficient video transcoding has become a core research topic for video technology developers. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides a video transcoding method, system, electronic device, and storage medium.

[0004] Firstly, this application provides a video transcoding method, the method comprising:

[0005] Acquire a video stream, decapsulate the video stream to obtain decapsulated data, the decapsulated data including the video bitstream;

[0006] Extract dynamic metadata from the video stream, traverse the dynamic metadata, and determine whether there is a scene switch;

[0007] When the scene switching is determined by traversing the dynamic metadata, the target dynamic metadata corresponding to the scene switching is recorded; the target frame number is determined based on the target dynamic metadata; and the scene switching frame and non-scene switching frame in the video stream are determined based on the target frame number.

[0008] During the re-encoding process, the encoding parameters of the scene switching frames and / or the non-scene switching frames are adjusted;

[0009] The video is encoded based on the adjusted encoding parameters to obtain the transcoded bitstream.

[0010] In one embodiment, determining the scene transition frames and non-scene transition frames in the video stream based on the target frame number includes:

[0011] Decode the video stream to obtain YUV data, read the YUV data frame by frame for re-encoding, and determine whether the read YUV data is the scene switching frame or the non-scene switching frame according to the target frame number;

[0012] The adjustment of the encoding parameters of the scene switching frame and / or the non-scene switching frame includes:

[0013] During re-encoding, if the read YUV data is a non-scene switching frame, then the read YUV data is determined to be a non-I-frame, and / or the qp value is increased.

[0014] During re-encoding, if the read YUV data is the scene switching frame, then the read YUV data is determined to be an I-frame, and / or the qp value is lowered.

[0015] In one embodiment, the step of performing video encoding processing based on the adjusted encoding parameters to obtain a transcoded bitstream includes:

[0016] If the read YUV data is the non-scene switching frame, then the read YUV data is encoded as a non-I frame, and / or encoded according to the increased qp value;

[0017] If the read YUV data is the scene switching frame, then the read YUV data is encoded as an I-frame, and / or encoded according to the lowered qp value;

[0018] The dynamic metadata is added, and the current frame data is encoded.

[0019] All encoded frame data are encapsulated according to a preset encapsulation format, and the transcoded bitstream is output.

[0020] In one embodiment, determining whether a scene switch exists includes:

[0021] Obtain the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame; the current frame is the Nth frame in the sliding window, and the target frame is each frame in the previous N-1 frames in the sliding window; the size of the sliding window is N, N≥2;

[0022] Based on the difference value and the preset difference threshold, it is determined whether the scene switching exists.

[0023] In one embodiment, the difference value includes a deviation value. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0024] The deviation value is calculated by substituting the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of the target frame into the first deviation calculation formula.

[0025] Alternatively, the dynamic metadata parameter value of the current frame, the dynamic metadata parameter value of the target frame, and the preset weight value can be substituted into the second deviation calculation formula to calculate the deviation value.

[0026] In one embodiment, the difference value includes a deviation value. When N≥3, the difference value obtained between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0027] Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the first deviation calculation formula to calculate multiple first deviation values; perform mean calculation or weighted calculation based on the multiple first deviation values ​​to obtain the deviation value.

[0028] Alternatively, the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula to calculate multiple second deviation values; the deviation values ​​are obtained by averaging or weighting the multiple second deviation values.

[0029] In one embodiment, the difference value includes curve deviation. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame may include the following steps:

[0030] Tone mapping is performed on the dynamic metadata of the current frame to obtain a first mapping curve;

[0031] Tone mapping is performed on the dynamic metadata of the target frame to obtain a second mapping curve;

[0032] Obtain the curve deviation between the first mapping curve and the second mapping curve.

[0033] In one embodiment, obtaining the curve deviation between the first mapping curve and the second mapping curve includes:

[0034] Determine the coordinate values ​​of the first mapping curve and the coordinate values ​​of the second mapping curve;

[0035] The curve deviation is calculated by substituting the coordinate values ​​of the first and second mapped curves into the preset curve deviation calculation formula.

[0036] In one embodiment, the difference value includes a relevance value. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0037] The relevance value is calculated by substituting the dynamic metadata parameter values ​​of the current frame and the target frame into the relevance calculation formula.

[0038] The difference value includes the relevance value. When N≥3, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0039] Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the relevance calculation formula to calculate multiple first relevance values; perform mean calculation or weighted calculation based on the multiple first relevance values ​​to obtain the relevance value.

[0040] In one embodiment, the preset difference threshold includes a preset deviation threshold, and determining whether the scene switch exists based on the difference value and the preset difference threshold includes:

[0041] Determine whether the deviation value is greater than or equal to the preset deviation threshold; if so, determine that the scene switching has occurred.

[0042] If not, then it is determined that the scene switch does not exist.

[0043] In one embodiment, the preset difference threshold includes a preset curve deviation threshold, and determining whether the scene switch exists based on the difference value and the preset difference threshold includes:

[0044] Determine whether the curve deviation is greater than or equal to the preset curve deviation threshold; if so, determine that the scene switching has occurred.

[0045] If not, then it is determined that the scene switch does not exist.

[0046] In one embodiment, the preset difference threshold includes a preset relevance threshold, and determining whether the scene switch exists based on the difference value and the preset difference threshold includes:

[0047] Determine whether the relevance value is less than or equal to the preset relevance threshold; if so, determine that the scene switching has occurred.

[0048] If not, then it is determined that the scene switch does not exist.

[0049] Secondly, this application provides a video transcoding system, the video transcoding system comprising:

[0050] An acquisition module is used to acquire a video stream, decapsulate the video stream, and obtain decapsulated data, wherein the decapsulated data includes the video bitstream.

[0051] The judgment module is used to extract dynamic metadata from the video stream, traverse the dynamic metadata, and determine whether there is a scene switch.

[0052] The processing module is configured to, when it is determined that a scene switch has occurred by traversing the dynamic metadata, record the target dynamic metadata corresponding to the scene switch; determine the target frame number based on the target dynamic metadata; and determine the scene switch frame and non-scene switch frame in the video stream based on the target frame number.

[0053] An adjustment module is used to adjust the encoding parameters of the scene switching frame and / or the non-scene switching frame during the re-encoding process;

[0054] The transcoding module is used to perform video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream.

[0055] Thirdly, this application provides an electronic device including a memory and a processor, the memory being used to store a computer program, the computer program executing the video transcoding method provided in the first aspect when the processor is running.

[0056] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, executes the video transcoding method provided in the first aspect.

[0057] The video transcoding method provided in this application involves acquiring a video stream, decapsulating the video stream to obtain decapsulated data, which includes a video bitstream; extracting dynamic metadata from the video bitstream, traversing the dynamic metadata to determine if a scene switch exists; when traversing the dynamic metadata determines that a scene switch exists, recording the target dynamic metadata corresponding to the scene switch; determining a target frame number based on the target dynamic metadata, and determining scene switch frames and non-scene switch frames in the video bitstream based on the target frame number; adjusting the encoding parameters of the scene switch frames and / or non-scene switch frames during re-encoding; and performing video encoding processing based on the adjusted encoding parameters to obtain a transcoded bitstream. In this way, dynamic metadata is used to identify video scene switches, and bitrate control is adjusted during video encoding based on scene switches. Non-scene switch frames reduce code bytes, saving total codewords and improving encoding performance. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation on the scope of protection of this application. In the various drawings, similar components are numbered similarly.

[0059] Figure 1 A first flowchart of the video transcoding method provided in this application is shown;

[0060] Figure 2 A second flowchart illustrating the video transcoding method provided in this application is shown;

[0061] Figure 3 A third flowchart of the video transcoding method provided in this application is shown;

[0062] Figure 4 A schematic diagram of the mapping curve provided in this application is shown;

[0063] Figure 5 The fourth flowchart of the video transcoding method provided in this application is shown;

[0064] Figure 6 The fifth flowchart of the video transcoding method provided in this application is shown;

[0065] Figure 7 A schematic diagram of the structure of the video transcoding system provided in this application is shown.

[0066] Icons: 700 - Video transcoding system, 701 - Acquisition module, 702 - Judgment module, 703 - Processing module, 704 - Adjustment module, 705 - Transcoding module. Detailed Implementation

[0067] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0068] The components of this application, typically described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0069] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0070] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0071] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0072] Example 1

[0073] This application provides a video transcoding method that can use dynamic metadata to determine scene switching and perform video transcoding based on the scene switching determination result, thereby improving video transcoding efficiency.

[0074] See Figure 1 Video transcoding methods include:

[0075] Step S101: Obtain the video stream, decapsulate the video stream to obtain decapsulated data, the decapsulated data including the video bitstream.

[0076] In this embodiment, the video stream can be a video stream containing dynamic metadata or a video stream without dynamic metadata. Video streams containing dynamic metadata include, but are not limited to, High Dynamic Range (HDR) video, HDR Vivid, HDR 10+, and Dolby Vision, and are not limited here. Other types of video streams can be Standard Dynamic Range (SDR) video.

[0077] Additionally, the decapsulation of the video stream includes: determining whether the video stream is in a preset encapsulation format; if the video stream is in a preset encapsulation format, then decapsulating the video stream to obtain the video bitstream. The preset encapsulation format can be MP4, TS, MOV, etc., and is not limited here.

[0078] It's understandable that encoded videos, such as h265, h266, avs2, and avs3, are encapsulated into encapsulation / transmission formats like MP4, TS, and MOV. Correspondingly, the process of extracting h265, h266, avs2, and avs3 Elasticsearch (ES) streams from MP4, TS, and MOV video streams is called decapsulation; that is, what is obtained after decapsulation is the encoded ES stream.

[0079] Step S102: Extract dynamic metadata from the video stream, traverse the dynamic metadata, and determine whether there is a scene switch.

[0080] In this embodiment, extracting dynamic metadata from the video stream may include the following steps: extracting NAL units frame by frame from the video stream, determining whether the extracted NAL units are of the NAL type that stores dynamic metadata; if so, extracting the dynamic metadata; otherwise, decoding to obtain YUV data.

[0081] See Figure 2 The determination of whether a scene switch exists in step S102 may include the following steps:

[0082] Step S1021: Obtain the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame; the current frame is the Nth frame in the sliding window, and the target frame is each frame in the previous N-1 frames in the sliding window; the size of the sliding window is N, N≥2;

[0083] Step S1022: Determine whether the scene switching exists based on the difference value and the preset difference threshold.

[0084] In one embodiment, the difference value includes a deviation value. When N is 2, step S1021 may include:

[0085] The deviation value is calculated by substituting the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of the target frame into the first deviation calculation formula.

[0086] Alternatively, the dynamic metadata parameter value of the current frame, the dynamic metadata parameter value of the target frame, and the preset weight value can be substituted into the second deviation calculation formula to calculate the deviation value.

[0087] It should be noted that the formula for calculating the first deviation is:

[0088] (1)

[0089] Where d1(x, y) represents the deviation value, x i This represents the dynamic metadata parameter values ​​of the current frame, and there are n values ​​in total, y i This represents the dynamic metadata parameter values ​​of the target frame, and the number of these parameters is n (x... i y i ) represents a dynamic metadata parameter pair, x i y i The parameter value represents the same as the dynamic metadata name.

[0090] It should be noted that the formula for calculating the second deviation is:

[0091] (2)

[0092] Where d2(x,y) represents the deviation value, x i This represents the dynamic metadata parameter values ​​of the current frame, and there are n values ​​in total, y i This represents the dynamic metadata parameter values ​​of the target frame, and the number of these parameters is n (x... i y i ) represents a dynamic metadata parameter pair, x i y i The parameter value that represents the same dynamic metadata name, w i The weight values ​​of different dynamic metadata, w i The larger the value, the more important the corresponding dynamic metadata. This weight value can be customized based on user experience or obtained through experiments using historical data.

[0093] In another embodiment, the difference value includes a deviation value. When N≥3, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0094] Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the first deviation calculation formula to calculate multiple first deviation values; perform mean calculation or weighted calculation based on the multiple first deviation values ​​to obtain the deviation value.

[0095] Alternatively, the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula to calculate multiple second deviation values; the deviation values ​​are obtained by averaging or weighting the multiple second deviation values.

[0096] It is understandable that the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame are substituted into the first deviation calculation formula, that is, after being substituted into formula (1), d1(x, y) is calculated, which represents the first deviation value.

[0097] It should be noted that the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula, that is, after being substituted into formula (2), d2(x, y) is calculated, which represents the second deviation value.

[0098] See Figure 3 The difference value includes curve deviation. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame may include the following steps:

[0099] Step S10211: Perform tone mapping on the dynamic metadata of the current frame to obtain the first mapping curve;

[0100] Step S10212: Perform tone mapping on the dynamic metadata of the target frame to obtain a second mapping curve;

[0101] Step S10213: Obtain the curve deviation between the first mapping curve and the second mapping curve.

[0102] In one embodiment, step S10213 includes the following steps:

[0103] Determine the coordinate values ​​of the first mapping curve and the coordinate values ​​of the second mapping curve;

[0104] The curve deviation is calculated by substituting the coordinate values ​​of the first and second mapped curves into the preset curve deviation calculation formula.

[0105] In one embodiment, the preset curve deviation calculation formula is:

[0106] (3)

[0107] Where S represents the curve deviation degree, Y1 j Y2 represents the j-th coordinate Y value of the first mapping curve. j This represents the j-th coordinate Y value of the second mapping curve.

[0108] It should be noted that a preset number of coordinate values ​​can be obtained from both the first and second mapping curves. The preset number can be customized; for example, it could be 200 or 1000, with no specific limitation. The selected coordinate values ​​include key coordinate values, such as the endpoints of linear splines and cubic splines.

[0109] See Figure 4 , Figure 4This is an example diagram of the HDR Vivid color mapping curve. The horizontal axis represents the brightness value before mapping (corresponding to a range of 0-10000 nits), normalized to 0-1 after photoelectric conversion. The vertical axis represents the brightness value after mapping (i.e., the output brightness value after adaptation to the target display device), including six line segments defined by the first interpolation point of interval 1, the second interpolation point of interval 1, the third interpolation point of interval 1, the first interpolation point of interval 2, the second interpolation point of interval 2, and the third interpolation point of interval 2. Specifically, the segment from the origin to the first interpolation point of interval 1 represents a linear spline interval. The segment from the first interpolation point of interval 1 to the second interpolation point of interval 1, plus the segment from the second interpolation point of interval 1 to the third interpolation point of interval 1, represents cubic spline interval 1. The segment from the third interpolation point of interval 1 to the first interpolation point of interval 2 represents the basic curve interval. The segment from the first interpolation point of interval 2 to the second interpolation point of interval 2, plus the segment from the second interpolation point of interval 2 to the third interpolation point of interval 2, represents cubic spline interval 2. It should be noted that the mapping curves have their own characteristics depending on the different metadata parameters, and are not necessarily the same. Figure 4 The six line segments shown have different mapping curves corresponding to different metadata, which are not limited here.

[0110] In one embodiment, the difference value includes curve difference. When N≥3, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame may include the following steps:

[0111] Tone mapping is performed on the dynamic metadata of the current frame to obtain a first mapping curve;

[0112] Tone mapping is performed on the dynamic metadata of each target frame to obtain multiple second mapping curves;

[0113] The first curve deviation degree between the first mapping curve and each of the second mapping curves is obtained respectively; the curve deviation degree is obtained by averaging or weighting the multiple first curve deviation degrees.

[0114] It should be noted that the deviation of the first curve between the first mapping curve and each of the second mapping curves can be obtained by substituting the coordinate values ​​of the first mapping curve and each of the second mapping curves into formula (3), and the calculated S represents the deviation of the first curve.

[0115] In one embodiment, the difference value includes a relevance value. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0116] The relevance value is calculated by substituting the dynamic metadata parameter values ​​of the current frame and the target frame into the relevance calculation formula.

[0117] In one embodiment, the difference value includes a relevance value. When N≥3, the difference value obtained between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0118] Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the relevance calculation formula to calculate multiple first relevance values; perform mean calculation or weighted calculation based on the multiple first relevance values ​​to obtain the relevance value.

[0119] It should be noted that the relevance calculation formula can be the cosine similarity calculation formula, the average matching similarity calculation formula, etc., and there are no restrictions here.

[0120] As an example, the formula for calculating cosine similarity can be the formula for calculating cosine similarity using the Discrete Cosine Transform (DCT), which can be specifically as follows:

[0121] (4)

[0122] in, These represent the DCT feature vectors of the current frame and each target frame, respectively. This represents the cosine similarity value for the Discrete Cosine Transform (DCT).

[0123] In one embodiment, the difference value includes a relevance value. When N≥3, the difference value obtained between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes:

[0124] Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the relevance calculation formula to calculate multiple first relevance values; perform mean calculation or weighted calculation based on the multiple first relevance values ​​to obtain the relevance value.

[0125] It should be noted that the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame can be substituted into the cosine similarity calculation formula or the average matching similarity calculation formula to calculate multiple first relevance values. By using the mean or first relevance values, a more accurate relevance value can be obtained.

[0126] In one embodiment, the preset difference threshold includes a preset deviation threshold, and step S1022 includes the following steps:

[0127] Determine whether the deviation value is greater than or equal to the preset deviation threshold; if so, determine that the scene switching has occurred.

[0128] If not, then it is determined that the scene switch does not exist.

[0129] In this embodiment, a larger deviation value indicates a higher degree of deviation between the dynamic metadata parameter values ​​of the current frame and the target frame, and a lower degree of similarity. The preset deviation threshold is determined based on actual conditions or empirical values. For example, the preset deviation threshold can be set to 0.025, 0.05, 0.01, etc., or it can be set to other values.

[0130] In one embodiment, the preset difference threshold includes a preset curve deviation threshold. Step S1022 includes the following steps:

[0131] Determine whether the curve deviation is greater than or equal to the preset curve deviation threshold; if so, determine that the scene switching has occurred.

[0132] If not, then it is determined that the scene switch does not exist.

[0133] It should be noted that a larger curve deviation indicates a higher degree of deviation between the first mapping curve of the current frame and the second mapping curve of the target frame, and a lower similarity. The preset curve deviation threshold can be determined based on experimental data; for example, it can be set to 0.025, 0.06, 0.09, etc., without any restrictions here.

[0134] In one embodiment, the preset difference threshold includes a preset correlation threshold. Step S1022 includes the following steps:

[0135] Determine whether the relevance value is less than or equal to the preset relevance threshold; if so, determine that the scene switching has occurred.

[0136] If not, then it is determined that the scene switch does not exist.

[0137] It should be noted that a higher relevance value indicates a higher similarity and lower deviation between the dynamic metadata of the current frame and the dynamic metadata of the target frame. The preset relevance threshold can be determined based on experimental data; for example, it can be set to 0.80, 0.85, 0.90, 0.975, etc., without any restrictions here.

[0138] Step S103: When the scene switching is determined by traversing the dynamic metadata, the target dynamic metadata corresponding to the scene switching is recorded; the target frame number is determined according to the target dynamic metadata; and the scene switching frame and non-scene switching frame in the video stream are determined according to the target frame number.

[0139] In this embodiment, determining the scene switching frames and non-scene switching frames in the video stream based on the target frame number includes:

[0140] The video stream is decoded to obtain YUV data. The YUV data is read frame by frame and re-encoded. The YUV data read is determined to be either the scene switching frame or the non-scene switching frame based on the target frame number.

[0141] It should be noted that the video bitstream is an ES stream such as h265 or h266, which is decoded into multi-frame YUV data. The YUV data is the original pixel data of the video.

[0142] Step S104: Adjust the encoding parameters of the scene switching frame and / or the non-scene switching frame.

[0143] See Figure 5 Step S104 includes:

[0144] Step S1041: During re-encoding, if the read YUV data is the non-scene switching frame, then the read YUV data is determined to be a non-I-frame, and / or the qp value is increased.

[0145] In step S1042, during re-encoding, if the read YUV data is the scene switching frame, then the read YUV data is determined to be an I-frame, and / or the qp value is lowered.

[0146] Additionally, qp represents the quantization parameter. The quantization parameter qp affects codewords and image quality. The lower the qp, the more codewords are allocated, resulting in a better image. Different encoders have different qp value ranges; for example, in H.265, it ranges from 0 to 51. I-frames are the frame type used in image encoding, while non-I-frames are the opposite of I-frames.

[0147] As an example, the processing of multi-frame YUV data is done frame by frame. The current frame number is counted, and if it's in the scene transition frame number list, it's determined to be a scene transition frame and encoded as an I-frame. Increasing the codeword count ensures image quality for scene transition frames. It's worth noting that I-frames have a high codeword count, so additional QP adjustments are unnecessary. Encoding as a non-I-frame can be achieved by simultaneously lowering the QP value. If the encoding method uses a fixed I-frame interval, non-scene transition frames that are not designated fixed I-frames can be encoded as non-I-frames or the QP value can be increased to reduce the codeword count, thus saving codewords.

[0148] It should be noted that existing transcoders supporting dynamic metadata primarily address image pixel size, position, scaling, cropping, rotation, and mirroring operations, ensuring the consistency of dynamic metadata throughout the transcoding process. In this invention, dynamic metadata is used to identify video scene transitions. Based on these scene transitions, bitrate control is adjusted during video encoding, making bitrate control more targeted. Exemplary bitrate adjustments include, but are not limited to, inserting I-frames in scene transition frames, lowering the qp (query point per second) in scene transition frames to increase codewords, and reducing codewords in non-scene transition frames, thereby saving total codewords and ultimately improving encoding performance.

[0149] Step S105: Perform video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream.

[0150] See Figure 6 Step S105 includes:

[0151] Step S1051: If the read YUV data is the non-scene switching frame, then the read YUV data is encoded as a non-I frame, and / or encoded according to the increased qp value.

[0152] Step S1052: If the read YUV data is the scene switching frame, then the read YUV data is encoded as an I-frame, and / or encoded according to the lowered qp value.

[0153] Step S1053: Add the dynamic metadata and encode it to obtain the current frame data.

[0154] It is understandable that in step S1053, dynamic metadata of the corresponding frame is added frame by frame, and after encoding, the data of each frame is obtained.

[0155] Step S1054: Encapsulate all encoded frame data according to a preset encapsulation format and output the transcoded bitstream.

[0156] As an example, using H.265, encoding is done frame by frame. A single frame of YUV data undergoes processes such as block division, prediction, transform quantization, and entropy coding to complete the encoding. Preset container formats include MP4, TS, and MOV, among others; no restrictions are imposed here.

[0157] The video transcoding method provided in this embodiment acquires a video stream, decapsulates the video stream to obtain decapsulated data, the decapsulated data including the video bitstream; extracts dynamic metadata from the video bitstream, traverses the dynamic metadata to determine if a scene switch exists; when traversing the dynamic metadata determines that a scene switch exists, records the target dynamic metadata corresponding to the scene switch; determines the target frame number based on the target dynamic metadata, and determines the scene switch frame and non-scene switch frame in the video bitstream based on the target frame number; during re-encoding, adjusts the encoding parameters of the scene switch frame and / or the non-scene switch frame; performs video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream. In this way, dynamic metadata is used to identify video scene switches, and bitrate control is adjusted during video encoding based on scene switches, reducing code bytes in non-scene switch frames to save total codewords and improve encoding performance.

[0158] Example 2

[0159] In addition, this application provides a video transcoding system.

[0160] Specifically, such as Figure 7 As shown, the video transcoding system 700 includes:

[0161] The acquisition module 701 is used to acquire a video stream, decapsulate the video stream, and obtain decapsulated data, wherein the decapsulated data includes the video bitstream.

[0162] The judgment module 702 is used to extract dynamic metadata from the video bitstream, traverse the dynamic metadata, and determine whether there is a scene switch.

[0163] Processing module 703 is configured to, when traversing the dynamic metadata and determining that a scene switch has occurred, record the target dynamic metadata corresponding to the scene switch; determine the target frame number based on the target dynamic metadata; and determine the scene switch frame and non-scene switch frame in the video stream based on the target frame number.

[0164] The adjustment module 704 is used to adjust the encoding parameters of the scene switching frame and / or the non-scene switching frame during the re-encoding process;

[0165] The transcoding module 705 is used to perform video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream.

[0166] In one embodiment, the processing module 703 is used to decode the video stream to obtain YUV data, read the YUV data frame by frame for re-encoding, and determine whether the read YUV data is the scene switching frame or the non-scene switching frame according to the target frame number.

[0167] The adjustment module 704 is used to determine the read YUV data as a non-I-frame and / or increase the qp value if the read YUV data is a non-scene switching frame during re-encoding.

[0168] During re-encoding, if the read YUV data is the scene switching frame, then the read YUV data is determined to be an I-frame, and / or the qp value is lowered.

[0169] In one embodiment, the transcoding module 705 is used to encode the read YUV data into a non-I-frame if the read YUV data is the non-scene switching frame, and / or to encode it according to the increased qp value;

[0170] If the read YUV data is the scene switching frame, then the read YUV data is encoded as an I-frame, and / or encoded according to the lowered qp value;

[0171] The dynamic metadata is added, and the current frame data is encoded.

[0172] All encoded frame data are encapsulated according to a preset encapsulation format, and the transcoded bitstream is output.

[0173] In one embodiment, the determination module 702 is used to obtain the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame; the current frame is the Nth frame in the sliding window, and the target frame is each frame in the previous N-1 frames in the sliding window; the size of the sliding window is N, where N≥2;

[0174] Based on the difference value and the preset difference threshold, it is determined whether the scene switching exists.

[0175] In one embodiment, the difference value includes a deviation value. When N is 2, the judgment module 702 is used to substitute the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of the target frame into the first deviation calculation formula to calculate the deviation value.

[0176] Alternatively, the dynamic metadata parameter values ​​of the current frame, the target frame, and the preset weight value can be substituted into the second deviation calculation formula to calculate the deviation value. In one embodiment, the difference value includes a deviation value. When N≥3, the judgment module 702 is used to substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the first deviation calculation formula to calculate multiple first deviation values; and to calculate the deviation value by averaging or weighting the multiple first deviation values.

[0177] Alternatively, the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula to calculate multiple second deviation values; the deviation values ​​are obtained by averaging or weighting the multiple second deviation values.

[0178] In one embodiment, the difference value includes curve deviation. When N is 2, the judgment module 702 is used to perform tone mapping on the dynamic metadata of the current frame to obtain a first mapping curve.

[0179] Tone mapping is performed on the dynamic metadata of the target frame to obtain a second mapping curve;

[0180] Obtain the curve deviation between the first mapping curve and the second mapping curve.

[0181] In one embodiment, the determination module 702 is used to determine the coordinate values ​​of the first mapping curve and the coordinate values ​​of the second mapping curve;

[0182] The curve deviation is calculated by substituting the coordinate values ​​of the first and second mapped curves into the preset curve deviation calculation formula.

[0183] In one embodiment, the difference value includes curve difference. When N≥3, the judgment module 702 is used to perform tone mapping on the dynamic metadata of the current frame to obtain a first mapping curve.

[0184] Tone mapping is performed on the dynamic metadata of each target frame to obtain multiple second mapping curves;

[0185] The first curve deviation between the first mapping curve and each of the second mapping curves is obtained respectively;

[0186] The curve deviation is obtained by averaging or weighting multiple first curve differences. In one embodiment, the difference value includes a correlation value. When N is 2, the judgment module 702 is used to substitute the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of the target frame into the correlation calculation formula to calculate the correlation value.

[0187] The difference value includes the relevance value. When N≥3, the judgment module 702 is used to substitute the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of each target frame into the relevance calculation formula to calculate multiple first relevance values; and to calculate the relevance value by averaging or weighting the multiple first relevance values.

[0188] In one embodiment, the preset difference threshold includes a preset deviation threshold. The judgment module 702 is used to determine whether the deviation value is greater than or equal to the preset deviation threshold. If so, it is determined that the scene switching exists.

[0189] If not, then it is determined that the scene switch does not exist.

[0190] In one embodiment, the preset difference threshold includes a preset curve deviation threshold. The judgment module 702 is used to determine whether the curve deviation is greater than or equal to the preset curve deviation threshold. If so, it is determined that the scene switching exists.

[0191] If not, then it is determined that the scene switch does not exist. In one embodiment, the preset difference threshold includes a preset relevance threshold, and the judgment module 702 is used in the scene switch, including:

[0192] Determine whether the relevance value is less than or equal to the preset relevance threshold; if so, determine that the scene switching has occurred.

[0193] If not, then it is determined that the scene switch does not exist.

[0194] The video transcoding system 700 provided in this embodiment can implement the video transcoding method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0195] The video transcoding system provided in this embodiment acquires a video stream, decapsulates the video stream to obtain decapsulated data, the decapsulated data including the video bitstream; extracts dynamic metadata from the video bitstream, traverses the dynamic metadata to determine if a scene switch exists; when traversing the dynamic metadata determines that a scene switch exists, records the target dynamic metadata corresponding to the scene switch; determines the target frame number based on the target dynamic metadata, and determines the scene switch frame and non-scene switch frame in the video bitstream based on the target frame number; adjusts the encoding parameters of the scene switch frame and / or the non-scene switch frame; and performs video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream. In this way, dynamic metadata is used to identify video scene switches, and bitrate control is adjusted during video encoding based on scene switches. Non-scene switch frames reduce code bytes, saving total codewords and improving encoding performance.

[0196] Example 3

[0197] In addition, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the computer program executes the video transcoding method provided in Embodiment 1 when running on the processor.

[0198] The electronic device provided in this embodiment can implement the video transcoding method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0199] Example 4

[0200] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the video transcoding method provided in Embodiment 1.

[0201] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0202] The computer-readable storage medium provided in this embodiment can implement the video transcoding method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0203] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0205] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other modifications under the guidance of this application without departing from its spirit, and all of these modifications are within the scope of protection of this application.

Claims

1. A video transcoding method, characterized in that, The method includes: Acquire a video stream, decapsulate the video stream to obtain decapsulated data, the decapsulated data including the video bitstream; Extract dynamic metadata from the video stream, traverse the dynamic metadata, and determine whether there is a scene switch; When the scene switching is determined by traversing the dynamic metadata, the target dynamic metadata corresponding to the scene switching is recorded; the target frame number is determined based on the target dynamic metadata; and the scene switching frame and non-scene switching frame in the video stream are determined based on the target frame number. During the re-encoding process, the encoding parameters of the scene switching frames and / or the non-scene switching frames are adjusted; The video is encoded based on the adjusted encoding parameters to obtain the transcoded bitstream. The extraction of dynamic metadata from the video stream includes: NAL units are extracted frame by frame from the video stream. It is determined whether the extracted NAL unit is of the NAL type that stores dynamic metadata. If so, the dynamic metadata is extracted; otherwise, the YUV data is decoded. The determination of whether a scene switch exists includes: Obtain the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame; the current frame is the Nth frame in the sliding window, and the target frame is each frame in the previous N-1 frames in the sliding window; the size of the sliding window is N, N≥2; Based on the difference value and the preset difference threshold, it is determined whether the scene switching exists; The difference value includes the deviation value. When N≥3, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes: Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the first deviation calculation formula to calculate multiple first deviation values; perform mean calculation or weighted calculation based on the multiple first deviation values ​​to obtain the deviation value. Alternatively, the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula to calculate multiple second deviation values; the deviation values ​​are then calculated by averaging or weighting the multiple second deviation values. The formula for calculating the first deviation is as follows: （1） The formula for calculating the second deviation is: （2） Where d1(x, y) represents the first deviation value, d2(x, y) represents the second deviation value, and x i This represents the dynamic metadata parameter values ​​of the current frame, and there are n values ​​in total, y i This represents the dynamic metadata parameter values ​​of the target frame, and the number of these parameters is n (x... i y i ) represents a dynamic metadata parameter pair, x i y i The parameter value that represents the same dynamic metadata name, w i w represents the weight value of different dynamic metadata. i The larger the value, the more important the corresponding dynamic metadata.

2. The method according to claim 1, characterized in that, The step of determining the scene switching frames and non-scene switching frames in the video stream based on the target frame number includes: Decode the video stream to obtain YUV data, read the YUV data frame by frame for re-encoding, and determine whether the read YUV data is the scene switching frame or the non-scene switching frame according to the target frame number; The adjustment of the encoding parameters of the scene switching frame and / or the non-scene switching frame includes: During re-encoding, if the read YUV data is a non-scene switching frame, then the read YUV data is determined to be a non-I-frame, and / or the qp value is increased. During re-encoding, if the read YUV data is the scene switching frame, then the read YUV data is determined to be an I-frame, and / or the qp value is lowered.

3. The method according to claim 2, characterized in that, The step of performing video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream includes: If the read YUV data is the non-scene switching frame, then the read YUV data is encoded as a non-I frame, and / or encoded according to the increased qp value; If the read YUV data is the scene switching frame, then the read YUV data is encoded as an I-frame, and / or encoded according to the lowered qp value; The dynamic metadata is added, and the current frame data is encoded. All encoded frame data are encapsulated according to a preset encapsulation format, and the transcoded bitstream is output.

4. The method according to claim 1, characterized in that, The difference value includes the deviation value. When N is 2, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes: The deviation value is calculated by substituting the dynamic metadata parameter value of the current frame and the dynamic metadata parameter value of the target frame into the first deviation calculation formula. Alternatively, the dynamic metadata parameter value of the current frame, the dynamic metadata parameter value of the target frame, and the preset weight value can be substituted into the second deviation calculation formula to calculate the deviation value.

5. The method according to claim 1, characterized in that, The difference value includes curve deviation. When N is 2, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes the following steps: Tone mapping is performed on the dynamic metadata of the current frame to obtain a first mapping curve; Tone mapping is performed on the dynamic metadata of the target frame to obtain a second mapping curve; Obtain the curve deviation between the first mapping curve and the second mapping curve.

6. The method according to claim 5, characterized in that, The step of obtaining the curve deviation between the first mapping curve and the second mapping curve includes: Determine the coordinate values ​​of the first mapping curve and the coordinate values ​​of the second mapping curve; The curve deviation is calculated by substituting the coordinate values ​​of the first and second mapped curves into the preset curve deviation calculation formula.

7. The method according to claim 1, characterized in that, The difference value includes curve deviation. When N≥3, obtaining the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes the following steps: Tone mapping is performed on the dynamic metadata of the current frame to obtain a first mapping curve; Tone mapping is performed on the dynamic metadata of each target frame to obtain multiple second mapping curves; The first curve deviation between the first mapping curve and each of the second mapping curves is obtained respectively; The curve deviation is obtained by averaging or weighting multiple first curve deviations.

8. The method according to claim 1, characterized in that, The difference value includes the relevance value. When N is 2, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes: The relevance value is calculated by substituting the dynamic metadata parameter values ​​of the current frame and the target frame into the relevance calculation formula. The difference value includes the relevance value. When N≥3, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes: Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the relevance calculation formula to calculate multiple first relevance values; perform mean calculation or weighted calculation based on the multiple first relevance values ​​to obtain the relevance value.

9. The method according to claim 4, characterized in that, The preset difference threshold includes a preset deviation threshold. Determining whether the scene switch exists based on the difference value and the preset difference threshold includes: Determine whether the deviation value is greater than or equal to the preset deviation threshold; if so, determine that the scene switching has occurred. If not, then it is determined that the scene switch does not exist.

10. The method according to any one of claims 5-7, characterized in that, The preset difference threshold includes a preset curve deviation threshold. Determining whether the scene switch exists based on the difference value and the preset difference threshold includes: Determine whether the curve deviation is greater than or equal to the preset curve deviation threshold; if so, determine that the scene switching has occurred. If not, then it is determined that the scene switch does not exist.

11. The method according to claim 8, characterized in that, The preset difference threshold includes a preset relevance threshold. Determining whether the scene switch exists based on the difference value and the preset difference threshold includes: Determine whether the relevance value is less than or equal to the preset relevance threshold; if so, determine that the scene switching has occurred. If not, then it is determined that the scene switch does not exist.

12. A video transcoding system, characterized in that, The system includes: An acquisition module is used to acquire a video stream, decapsulate the video stream, and obtain decapsulated data, wherein the decapsulated data includes the video bitstream. The judgment module is used to extract dynamic metadata from the video stream, traverse the dynamic metadata, and determine whether there is a scene switch. The processing module is configured to, when it is determined that a scene switch has occurred by traversing the dynamic metadata, record the target dynamic metadata corresponding to the scene switch; determine the target frame number based on the target dynamic metadata; and determine the scene switch frame and non-scene switch frame in the video stream based on the target frame number. An adjustment module is used to adjust the encoding parameters of the scene switching frame and / or the non-scene switching frame during the re-encoding process; The transcoding module is used to perform video encoding processing based on the adjusted encoding parameters to obtain the transcoded bitstream; The extraction of dynamic metadata from the video stream includes: NAL units are extracted frame by frame from the video stream. It is determined whether the extracted NAL unit is of the NAL type that stores dynamic metadata. If so, the dynamic metadata is extracted; otherwise, the YUV data is decoded. The determination of whether a scene switch exists includes: Obtain the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame; the current frame is the Nth frame in the sliding window, and the target frame is each frame in the previous N-1 frames in the sliding window; the size of the sliding window is N, N≥2; Based on the difference value and the preset difference threshold, it is determined whether the scene switching exists; The difference value includes the deviation value. When N≥3, the difference value between the dynamic metadata of the current frame and the dynamic metadata of the target frame includes: Substitute the dynamic metadata parameter values ​​of the current frame and the dynamic metadata parameter values ​​of each target frame into the first deviation calculation formula to calculate multiple first deviation values; perform mean calculation or weighted calculation based on the multiple first deviation values ​​to obtain the deviation value. Alternatively, the dynamic metadata parameter values ​​of the current frame, the dynamic metadata parameter values ​​of each target frame, and the preset weight values ​​are substituted into the second deviation calculation formula to calculate multiple second deviation values; the deviation values ​​are then calculated by averaging or weighting the multiple second deviation values. The formula for calculating the first deviation is as follows: （1） The formula for calculating the second deviation is: （2） Where d1(x, y) represents the first deviation value, d2(x, y) represents the second deviation value, and x i This represents the dynamic metadata parameter values ​​of the current frame, and there are n values ​​in total, y i This represents the dynamic metadata parameter values ​​of the target frame, and the number of these parameters is n (x... i y i ) represents a dynamic metadata parameter pair, x i y i The parameter value that represents the same dynamic metadata name, w i w represents the weight value of different dynamic metadata. i The larger the value, the more important the corresponding dynamic metadata.

13. An electronic device, characterized in that, The device includes a memory and a processor, the memory storing a computer program that executes the video transcoding method according to any one of claims 1 to 11 when the processor is running.

14. A computer-readable storage medium, characterized in that, It stores a computer program that, when run on a processor, executes the video transcoding method according to any one of claims 1 to 11.

Images