Method and device for transcoding input video, computing device, and storage medium
By automatically identifying video standards and adopting resolution and tone mapping processing, the complexity and picture quality problems of ultra-high-definition video transcoding in the existing technology are solved, and efficient video transcoding and picture quality assurance in the live broadcast field is achieved.
Patent Information
- Application Number
- PCT/CN2024/088052
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-11
- Filing Date
- 2024-04-16
- Publication Date
- 2024-11-14
AI Technical Summary
The existing video transcoding scheme increases complexity when supporting ultra-high-definition video, and the converted video quality may have distortion problems, especially when terminal devices do not support ultra-high-definition video in the live broadcast field.
By automatically identifying the standards of input videos, deciding the conversion method, using resolution, frame rate and tone mapping processing to reduce the complexity of transcoding services, ensuring picture quality, and supporting the conversion of ultra-high-definition video to high-definition or standard-definition.
It reduces the complexity of video transcoding services and ensures the converted video quality, especially during the conversion process from ultra-high definition to high definition or standard definition, avoids color distortion problems.
Smart Images

Figure CN2024088052_14112024_PF_FP_ABST
Abstract
Description
Method and device for transcoding input video, computing device and storage medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 11, 2023, with application number 202310537227.9 and invention name “Method and device, computing device and storage medium for transcoding input video”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present disclosure relates to the field of video, and in particular to video transcoding. Background Art
[0003] With the continuous evolution of video capture and playback devices, video resolution, frame rate, dynamic range, and other aspects have significantly improved. This has led to the development of a variety of video standards, from early standard-definition video to high-definition video, and more recently, ultra-high-definition video. Traditionally, live streaming only supported transcoding between standard-definition and high-definition video. In recent years, the increasing use of ultra-high-definition video in live streaming, particularly broadcast and television live streaming, has necessitated the addition of support for ultra-high-definition video standards to existing live streaming architectures, often increasing the complexity of live transcoding services.
[0004] Therefore, an improved video transcoding solution is needed.
[0005] Summary of the Invention
[0006] A technical problem to be solved by the present disclosure is to provide an improved video transcoding solution.
[0007] According to a first aspect of the present disclosure, a method for transcoding an input video is provided, comprising: automatically identifying a standard of the input video; and performing transcoding processing on the input video when the standard of the input video is different from the standard of an output video.
[0008] According to a second aspect of the present disclosure, a device for transcoding an input video is provided, comprising: an identification module configured to automatically identify a standard of the input video; and a transcoding module configured to perform transcoding processing on the input video when the standard of the input video is different from the standard of an output video.
[0009] According to a third aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method described in the first aspect above.
[0010] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising an executable code. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described in the first aspect above.
[0011] According to a fifth aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.
[0013] FIG1 shows a schematic flowchart of a method for transcoding an input video according to an embodiment of the present disclosure.
[0014] FIG2 shows a schematic block diagram of a device for transcoding an input video according to an embodiment of the present disclosure.
[0015] FIG3 shows a schematic structural diagram of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0016] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0017] In the present disclosure, video transcoding refers to converting a video (such as a video file or a video stream, etc.) from one standard to another, which may include any suitable processing for converting a video from a source standard to a target standard, such as the conversion processing of image quality standards and / or the conversion processing of coding standards, which will be described in detail later.
[0018] Existing video transcoding solutions require users to correctly select the input video and desired output video standards before transcoding to achieve the correct transcoding method. Furthermore, the need to combine and maintain fixed transcoding schemes for various input and output video standards complicates transcoding solution implementation. Using the wrong transcoding method can lead to issues with video color and quality, increasing the complexity of transcoding services and the potential for problems.
[0019] For example, in the live broadcast field, since many terminal devices currently do not support the playback of ultra-high-definition videos, the source video shot using the ultra-high-definition standard often needs to be converted to other standards, such as high-definition, standard-definition, etc. Live broadcast transcoding services need to support different links: ultra-high-definition to ultra-high-definition, ultra-high-definition to high-definition or standard-definition, and high-definition or standard-definition to high-definition or standard-definition. Compared with the previous link of only high-definition or standard-definition to high-definition or standard-definition, the resulting increase in link complexity is significant. In addition, the existing ultra-high-definition video to high-definition or standard-definition conversion solution used in live broadcasts cannot guarantee the quality of the converted video, and may cause problems such as obvious color distortion after conversion.
[0020] To address at least one of the aforementioned issues, the present disclosure proposes a video transcoding solution that automatically identifies the input video standard and uses this information to determine the conversion method, thereby reducing the complexity of the entire transcoding service. Furthermore, in certain specific embodiments, the present disclosure also proposes an improved transcoding process when converting between video quality standards, thereby ensuring the quality of the converted video.
[0021] The solution of the present disclosure can automatically select a video transcoding method by automatically identifying the standard of the input video, thereby reducing the complexity of related services. In addition, in some specific embodiments of the present disclosure, tone mapping processing can also be used to better support downconversion processing of image quality standards (i.e., conversion of high-quality standard video to low-quality standard video, for example, conversion of ultra-high-definition video to high-definition, standard-definition, and other standard video), thereby ensuring picture quality (hereinafter referred to as "image quality").
[0022] At least some embodiments of the scheme of the present disclosure will be described in detail below with reference to Figures 1 and 2. Those skilled in the art will understand that although the above discussion is about the transcoding problem of ultra-high-definition video in the live broadcast field, the scheme of the present disclosure is not limited to this, but can be applied to various fields requiring video transcoding and transcoding between videos of various different standards. The video standards mentioned in this disclosure are not limited to the aforementioned picture quality standards (such as ultra-high definition, high definition, standard definition, etc.), and can also include coding standards, etc., so that the transcoding of the video can also include conversion between different coding standards.
[0023] FIG1 shows a schematic flowchart of a method for transcoding an input video according to an embodiment of the present disclosure.
[0024] As shown in FIG. 1 , in step S110 , the standard of the input video is automatically identified.
[0025] In some embodiments of the present disclosure, video standards may include image quality standards primarily determined based on image quality indicators, where the image quality indicators may include resolution, frame rate, dynamic range, color gamut, and / or bit depth. In other words, video standards may include image quality standards determined based on resolution, frame rate, dynamic range, color gamut, and / or bit depth. Image quality standards may include multiple image quality levels, such as ultra-high-definition video, high-definition video, and standard-definition video.
[0026] For example, ultra-high-definition video may include video with a resolution of 4K or above, a frame rate of 30 fps (frames per second) or above, and a high dynamic range (HDR).
[0027] High-definition video may include video with a resolution of 1080p or above, a frame rate of 25fps or above, and a dynamic range of standard dynamic range (SDR).
[0028] Standard definition video may include video with a resolution of 720p or above, a frame rate of 25fps or above, and an SDR dynamic range.
[0029] As mentioned above, the video standards disclosed in the present invention are not limited to image quality standards, but may also include standards in other aspects, such as coding standards. That is, for example, in some embodiments of the present invention, the coding standard of the input video can be automatically identified, and the input video can be subjected to encoding-related transcoding processing when it is different from the coding standard of the desired output video. The image quality standard and / or coding standard of the input video can be automatically identified as needed, and the input video can be subjected to corresponding transcoding processing when there is a difference from the corresponding standard of the output video. For example, even if the image quality standards of the input video and the output video are the same (for example, both are ultra-high-definition videos), transcoding processing must be performed when the coding standards of the two are different, but there is no need to perform up / down conversion processing in terms of image quality.
[0030] In some embodiments, the standard of the input video can be automatically identified by parsing the metadata of the input video to obtain information about the resolution, frame rate, dynamic range, color gamut and / or bit depth of the input video; and determining the standard of the input video based on the obtained information.
[0031] In some embodiments, the metadata of the video may include metadata directly added when the video is shot (e.g., generated and added by a camera during shooting) and / or metadata subsequently added by other means. The metadata may include technical metadata describing technical aspects, such as image quality indicators such as resolution, frame rate, encoding, dynamic range, color gamut, and / or bit depth, and / or other information that can indirectly determine these image quality indicators, such as lens model, focal length, white balance, camera hardware serial number, lens hardware serial number, etc. The metadata of the video may be embedded in the video file and / or stored in a file other than the video file.
[0032] In some embodiments, the desired output video standard or related information (e.g., one or more of the aforementioned image quality metrics) may have been entered or set before transcoding begins. This standard, or a standard determined based on the relevant information, may then be used directly during the subsequent transcoding process. For example, a user may invoke the transcoding service by selecting a preset template or setting a customized template, in which the desired output video standard or related information (e.g., one or more of the aforementioned image quality metrics) is selected.
[0033] Alternatively, in other embodiments, in parallel with, before, or after the automatic identification step S110, the desired output video standard may be obtained through various means. For example, information about the destination terminal device to which the video is to be output may be obtained, and the output video standard may be determined based on the video standards supported by the destination terminal device. For example, information about the transmission link for the output video may be obtained to adaptively determine the output video standard. In other words, in some cases, the user does not need to set or specify the desired output video standard; instead, the appropriate output video standard may be automatically determined based on various information.
[0034] In some embodiments, a single field may be used to indicate the video standard. For example, different values of this field may be used to indicate whether the video standard is ultra-high definition, high definition, or standard definition. Alternatively, in other embodiments, depending on the actual application, only one or more image quality indicators may be used to indicate the video standard. For example, resolution, frame rate, and / or dynamic range information may be used to indicate whether the video standard is ultra-high definition, high definition, or standard definition.
[0035] After the standard of the input video is automatically identified, as shown in FIG. 1 , in step S120 , if the standard of the input video is different from the standard of the output video, a transcoding process is performed on the input video.
[0036] In some embodiments, when the image quality standards of the input video and the output video are different, up-transcoding or down-transcoding processing can be performed on the input video in terms of image quality as needed. For example, resolution conversion processing, frame rate conversion processing and / or tone mapping processing can be performed on the input video as needed. The resolution conversion processing can convert the resolution of the input video to the required resolution of the output video, the frame rate conversion processing can convert the frame rate of the input video to the frame rate of the output video, and the tone mapping processing can map the brightness within the dynamic range of the input video to the brightness within the dynamic range of the output video. In addition, in some embodiments, the tone mapping processing can generally achieve the conversion of dynamic range, color gamut and bit depth at the same time. In some embodiments, the resolution conversion processing can include a conversion processing based on interpolation. In some embodiments, the frame rate conversion processing can include a frame rate conversion processing achieved by dropping or repeating frames. In some embodiments, the tone mapping processing can include a conversion processing that maps the brightness within HDR to the brightness within SDR based on a brightness mapping curve.
[0037] For example, if the input video is UHD and the output video standard is also UHD, no image quality conversion will be performed on the input video. If the input video is UHD and the output video standard is HD or SD, the input UHD video can be down-converted in terms of resolution, frame rate, and / or dynamic range, while maintaining the output video quality. If the input video is HD / SD and the output video standard is also HD / SD, the image quality conversion can also be skipped, while maintaining the output quality.
[0038] Furthermore, the present disclosure notes that existing down-conversion solutions from UHD to HD / SD cannot guarantee the quality of the converted image, resulting in issues such as noticeable color distortion. The present disclosure has found that these issues are primarily caused by a lack of appropriate conversion of the dynamic range.
[0039] In some cases, the UHD standard differs significantly from the HD / SD standard, mainly in the following five dimensions: resolution, frame rate, dynamic range, color gamut, and bit depth. It is expected that the UHD video can be converted to the HD / SD video in these five dimensions. For example, the transcoding process performed on the UHD video can generally include resolution conversion processing and frame rate conversion processing, thereby converting the resolution and frame rate of the UHD video to the required resolution and frame rate of the HD / SD video. However, the prior art does not perform a corresponding conversion on the dynamic range, which may lead to problems such as obvious color distortion. In order to ensure the picture quality of the output video, the present disclosure proposes to use tone mapping processing to convert the dynamic range during the conversion process. In some cases, the tone mapping process not only converts the dynamic range but also better converts the color gamut. In this way, the picture quality can be guaranteed while achieving standard conversion.
[0040] These solutions disclosed herein are not limited to the aforementioned conversion from ultra-high definition to high definition / standard definition, but may also be applicable to other situations requiring dynamic range conversion. In some embodiments, when the dynamic range of the image quality standard of the input video is greater than the dynamic range of the image quality standard of the output video, the transcoding process performed on the input video may include at least a tone mapping process, wherein the tone mapping process maps the luminance within the dynamic range of the input video to the luminance within the dynamic range of the output video. In some embodiments, the tone mapping process may map the luminance within the dynamic range of the input video to the luminance within the dynamic range of the output video based on at least a luminance mapping curve that is preset or calculated based on the input video.
[0041] In some embodiments, the specific implementation of the above-mentioned resolution conversion processing, frame rate conversion processing and / or tone mapping processing can be selected based on the comprehensive image quality effect and efficiency as needed.
[0042] In some embodiments, the transcoding method described above supports real-time transcoding, i.e., the processing time per frame is less than the inverse of the frame rate. Thus, the method according to the present disclosure can also be used in the live broadcast field to support real-time transcoding of ultra-high-definition video, etc. In some embodiments, the transcoding method described above can be deployed in the cloud.
[0043] Several examples of methods for transcoding input videos according to the present disclosure have been described in detail above with reference to the accompanying drawings. These methods can automatically identify the input video's standard and automatically select a video transcoding method, thereby reducing the complexity of related services. Furthermore, in certain specific embodiments, tone mapping can be used to better support downconversion between image quality standards, thereby ensuring image quality.
[0044] An example of a transcoding device that can implement the above-mentioned method of the present disclosure will be described below with reference to FIG. 2 .
[0045] FIG2 shows a schematic block diagram of a transcoding device for transcoding an input video into a desired output video according to an embodiment of the present disclosure.
[0046] As shown in FIG. 2 , the transcoding device 200 may include an identification module 210 and a transcoding module 220 .
[0047] In some embodiments, the recognition module 210 may receive at least a portion of the input video data (e.g., metadata) and automatically identify the standard of the input video. The recognition module 210 then outputs the identified standard S1 of the input video to the transcoding module 220 for selecting an appropriate transcoding method.
[0048] As shown in FIG. 2 , in some embodiments, the transcoding module 220 may further receive a desired output video standard S2 and, when the input video standard S1 is different from the output video standard S2 , perform transcoding on the received input video.
[0049] In some embodiments, the transcoding module 220 may include a resolution conversion processing module 221, a frame rate conversion processing module 222, and a tone mapping processing module 223, thereby supporting at least video conversion of various image quality standards, including conversion of ultra-high-definition video to high-definition / standard-definition video. The transcoding module 220 may call these modules as needed to perform corresponding transcoding processing. In addition, the transcoding module 220 may also include other conversion processing modules not shown, such as a coding format conversion module, which may be used alone or in conjunction with other modules to perform corresponding transcoding processing as needed.
[0050] In some implementations, the transcoding device 200 may be implemented in the cloud, for example, by a container or container cluster on the cloud as needed.
[0051] FIG3 shows a schematic structural diagram of a computing device that can be used to implement the above-mentioned method for transcoding an input video according to an embodiment of the present disclosure.
[0052] 3 , computing device 300 includes a memory 310 and a processor 320 .
[0053] Processor 320 may be a multi-core processor or may include multiple processors. In some embodiments, processor 320 may include a general-purpose main processor and one or more specialized coprocessors, such as a graphics processing unit (GPU) or a digital signal processor (DSP). In some embodiments, processor 320 may be implemented using customized circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).
[0054] The memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 320 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all instructions and data required by the processor during operation. In addition, the memory 310 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 310 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.
[0055] The memory 310 stores executable codes. When the executable codes are processed by the processor 320 , the processor 320 can execute the above-mentioned method for transcoding the input video.
[0056] The method for transcoding an input video and the corresponding system or device according to the present disclosure have been described above in detail with reference to the accompanying drawings.
[0057] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0058] In addition, the method according to the present disclosure may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present disclosure.
[0059] Alternatively, the present disclosure may also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor is caused to perform the various steps of the above-mentioned method according to the present disclosure.
[0060] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.
[0061] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems and methods according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0062] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for transcoding an input video, comprising: Automatically identify the standard of input video; as well as In a case where the standard of the input video is different from the standard of the output video, a transcoding process is performed on the input video.
2. The method according to claim 1, wherein: The standard of the input video and the standard of the output video include image quality standards determined according to resolution, frame rate, dynamic range, color gamut and / or bit depth.
3. The method according to claim 1, wherein: The steps to automatically identify the standard of input video include: Parsing metadata of the input video to obtain information about resolution, frame rate, dynamic range, color gamut and / or bit depth of the input video; and The standard of the input video is determined according to the acquired information.
4. The method according to claim 2, wherein: In a case where the dynamic range in the image quality standard of the input video is greater than the dynamic range in the image quality standard of the output video, the transcoding process performed on the input video includes at least a tone mapping process, The tone mapping process maps the brightness within the dynamic range of the input video to the brightness within the dynamic range of the output video.
5. The method according to claim 4, wherein: The tone mapping process maps the brightness within the dynamic range of the input video to the brightness within the dynamic range of the output video based on at least a brightness mapping curve that is preset or calculated according to the input video.
6. The method according to claim 4, wherein: The transcoding process performed on the input video further includes: resolution conversion processing and / or frame rate conversion processing, wherein the resolution conversion process converts the resolution of the input video into the resolution of the output video, The frame rate conversion process converts the frame rate of the input video into the frame rate of the output video.
7. A device for transcoding an input video, comprising: a recognition module configured to automatically recognize a standard of an input video; as well as The transcoding module is configured to, when the standard of the input video is different from the standard of the output video, The input video is transcoded.
8. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 6.
9. A computer program product, comprising executable codes, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any one of claims 1 to 6.
10. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Video transcoding method, device and system and storage medium
CN112188285A
Video stream format conversion method and device and storage medium
CN113973214A
Method and device for transcoding input video, computing device and storage medium
CN116847102A
Display apparatus
JP2008107700A
Format converting apparatus and method
US20040218094A1
