Video alignment method and apparatus

Abstract

This application discloses a video alignment method and apparatus. The method includes: acquiring supplemental enhancement information (SEI) from a source video; transcoding the source video and copying the SEI information to it to obtain a transcoded video; and aligning the source video and the transcoded video based on the same SEI information in both. Copying the SEI information from the source video to the transcoded video ensures that the SEI information in the transcoded video is the same as that in the source video. By using the same SEI information in the source video and the transcoded video, video alignment can be performed without decoding the video frames of both videos, saving resources and significantly improving the processing speed of video alignment.

Description

Technical Field

[0001] This application relates to the field of live streaming technology, specifically to a video alignment method and apparatus. Background Technology

[0002] In live streaming, to facilitate user viewing and reduce video buffering, the streamer's original video can be transcoded to obtain a transcoded video, which can then be streamed to users.

[0003] For transcoded videos, quality assessment is necessary to ensure that the quality does not affect the user's viewing experience. Quality assessment can be performed by visually observing the same frames of the source and transcoded videos. However, due to the latency introduced during transcoding, the frames of the transcoded and source videos may not align. Therefore, a method for video alignment is needed. Summary of the Invention

[0004] In view of the above problems, embodiments of this application are proposed to provide a video alignment method and apparatus that overcomes or at least partially solves the above problems.

[0005] According to a first aspect of the embodiments of this application, a video alignment method is provided, comprising:

[0006] Obtain the Supplemental Enhancement Information (SEI) from the source video;

[0007] The source video is transcoded, and the SEI information is copied and written to obtain the transcoded video;

[0008] Based on the same SEI information in the source video and the transcoded video, the source video and the transcoded video are aligned.

[0009] Optionally, obtaining SEI information from the source video further includes:

[0010] The source video is decapsulated to obtain a structure that stores compressed encoded data;

[0011] Parse the structure to obtain the SEI information of the source video.

[0012] Optionally, after obtaining the SEI information from the source video, the method further includes:

[0013] The SEI information is stored in the cache based on the decoding timestamp of the structure.

[0014] Optionally, the source video is transcoded, and the SEI information is copied and written to obtain the transcoded video, which further includes:

[0015] Transcode the source video;

[0016] Retrieve the corresponding SEI information from the cache based on the decoding timestamp of the structure;

[0017] The SEI information is copied and written into the transcoded video to obtain the transcoded video.

[0018] Optionally, the method further includes:

[0019] Add SEI information to the video frames in the source video.

[0020] Optionally, the SEI information can be set in an incremental manner.

[0021] Optionally, the SEI information includes the SEI serial number, display timestamp, and / or identification code.

[0022] Optionally, video alignment of the source video and the transcoded video based on the same SEI information in the source video and the transcoded video further includes:

[0023] Decapsulate the source video and the transcoded video to obtain the SEI information of the source video and the SEI information of the transcoded video respectively;

[0024] By comparing the SEI information of the source video and the SEI information of the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information are determined.

[0025] Simultaneously play and / or store the first video frame of the source video and the second video frame of the transcoded video to complete the video alignment of the source video and the transcoded video.

[0026] Optionally, comparing the SEI information of the source video and the SEI information of the transcoded video to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information further includes:

[0027] By comparing the SEI number of the source video and the SEI number of the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI number are determined.

[0028] or,

[0029] Compare the display timestamp of the source video with the display timestamp of the transcoded video to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same display timestamp;

[0030] or,

[0031] By comparing the identifier codes of the source video and the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same identifier code are determined.

[0032] Optionally, the number of the first video frames of the source video and the number of the second video frames of the transcoded video are the same.

[0033] Optionally, the number of first video frames can be multiple; the number of second video frames can also be multiple.

[0034] Simultaneously playing and / or storing the first video frame of the source video and the second video frame of the transcoded video, the video alignment of the source video and the transcoded video further includes:

[0035] The first video frame of the source video and the second video frame of the transcoded video with the same SEI information are sorted according to the order of video playback and a one-to-one correspondence is established.

[0036] Play and / or store the first and second video frames that have a corresponding relationship simultaneously to complete the video alignment of the source video and the transcoded video.

[0037] According to a second aspect of the embodiments of this application, a video alignment apparatus is provided, comprising:

[0038] The acquisition module is suitable for acquiring the supplementary enhancement information (SEI) from the source video.

[0039] The copy-write module is suitable for transcoding the source video and copying and writing the SEI information to obtain the transcoded video;

[0040] The alignment module is suitable for aligning source and transcoded videos based on the same SEI information in the source and transcoded videos.

[0041] According to a third aspect of the embodiments of this application, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0042] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above video alignment method.

[0043] According to a fourth aspect of the present application, a computer storage medium is provided, the storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the video alignment method described above.

[0044] According to the video alignment method and apparatus provided in this application, the SEI information of the source video is copied and written into the transcoded video, meaning that the SEI information in the transcoded video and the source video is the same. Based on the identical SEI information in the source video and the transcoded video, video alignment can be performed without decoding the video frames of the source video and the transcoded video, saving resources and significantly improving the processing speed of video alignment.

[0045] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0046] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0047] Figure 1 A flowchart of a video alignment method according to an embodiment of this application is shown;

[0048] Figure 2 A flowchart of a video alignment method according to another embodiment of this application is shown;

[0049] Figure 3 A schematic diagram of the structure of a video alignment device according to an embodiment of this application is shown;

[0050] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of this application is shown. Detailed Implementation

[0051] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0052] First, the terms and concepts involved in one or more embodiments of this application will be explained.

[0053] Transcoding: Re-encoding audio and video;

[0054] Video alignment: The content played in the video is the same before and after transcoding;

[0055] Video codec: A program or device that can compress or decompress digital video;

[0056] Bitstream: A data structure that describes the attributes of video frames;

[0057] ffprobe: Collects information from multimedia streams and prints it out in a human- and machine-readable format;

[0058] ffmpeg: An open-source computer program that can be used to record and convert digital audio and video, and convert them into streams;

[0059] SEI (Supplemental Enhancement Information): Supplemental enhancement information provides users with methods to add additional information to the video stream;

[0060] PTS (Presentation Time Stamp): Displays a timestamp, telling the player when to play the data for this frame.

[0061] Figure 1 A flowchart of a video alignment method according to an embodiment of this application is shown, as follows: Figure 1 As shown, the method includes the following steps:

[0062] Step S101: Obtain the supplementary enhancement information (SEI) from the source video.

[0063] SEI information, or Supplemental Enhancement Information, can be inserted into audio and video streams to convey additional information. SEI information includes features such as payloadType (defining the SEI information type), payloadSize (the size of the SEI message), and content written starting from byte 16 in uuid_iso_iec_11578. This content can be customized. In this embodiment, SEI information is used for video alignment, and the content can use data such as tags for easy comparison. Video alignment is completed based on the comparison results. The specific content can be set according to the implementation situation and is not limited here.

[0064] Furthermore, SEI information is not a mandatory option in the decoding process. It can be used for error tolerance and correction during decoding and can be integrated into the video bitstream. In other words, SEI information can be inserted during video generation and transmission, transmitted along with the video through the transmission link, and can be obtained without decoding, reducing resource consumption and increasing processing speed. Based on these characteristics, before the edge computing node transcodes the source video, i.e., after the edge computing node receives the source video, the SEI information in the source video can be directly and quickly obtained. Specifically, this can be achieved by using ffmpeg to decapsulate the source video, obtaining the SEI information, determining the SEI information based on payloadType, payloadSize, etc., and reading the content contained in uuid_iso_iec_11578, etc.

[0065] Step S102: Transcode the source video and copy and write the SEI information to obtain the transcoded video.

[0066] Transcoding the source video requires first decoding and then re-encoding it. This process can utilize video codecs for decoding and encoding, facilitating streaming for users. Considering that SEI information can be modified during transcoding, leading to inconsistencies between the SEI information in the final transcoded video and the original video, this embodiment first obtains the SEI information from the source video, then transcodes it. After transcoding, the obtained SEI information is copied and written into the transcoded video, ensuring that the SEI information in the final transcoded video matches that of the original video.

[0067] Step S103: Align the source video and the transcoded video according to the same SEI information in the source video and the transcoded video.

[0068] After obtaining the transcoded video, the SEI information in the source and transcoded videos can be compared. This can be done by decapsulating both the source and transcoded videos to obtain their respective SEI information. Since SEI information is bound to video frames, the SEI information of the source and transcoded videos can be compared frame-by-frame to identify video frames with the same SEI information. These frames contain the same content, and the source and transcoded videos are then aligned to ensure they play the same content.

[0069] According to the video alignment method provided in this application, the SEI information of the source video is copied and written into the transcoded video, meaning that the SEI information in the transcoded video and the source video is the same. Based on the identical SEI information in the source and transcoded videos, video alignment can be performed without decoding the video frames of either video, saving resources and significantly improving the processing speed of video alignment.

[0070] Figure 2 A flowchart of a video alignment method according to an embodiment of this application is shown, as follows: Figure 2 As shown, the method includes the following steps:

[0071] Step S201: Add SEI information to the video frames in the source video.

[0072] SEI information can be added to the source video of a live stream. The SEI information can be added before the source video is transcoded, such as when the source video of the streamer is uploaded to the server, and the source video with the added SEI information is transmitted to the edge computing node for transcoding, or the source video is transmitted to the edge computing node and the SEI information is added before transcoding, etc. There are no restrictions here.

[0073] The added SEI information corresponds to the video frames in the source video. Specifically, SEI information is added to each video frame in the source video. SEI information is set in an incremental manner. The content contained in uuid_iso_iec_11578 can be the SEI sequence number sei_idx. In the first frame, the SEI sequence number sei_idx is set to 1, in the second frame it is set to 2, in the third frame it is set to 3, and so on. Based on the SEI sequence number, each video frame can be effectively distinguished, and the incremental order of the SEI sequence number corresponds to the order of the video frames. Alternatively, the content contained in uuid_iso_iec_11578 can be the PTS (Presentation Time Stamp). The PTS tells the video player the specific time to play its corresponding video frame. The PTS is related to the order of the video frames in the source video. The PTS time of the video frame that comes first is earlier than the PTS time of the video frame that comes later. The PTS in each video frame will also show an incremental trend according to the order of the video frames. The specific value of PTS can be set by the encoder when generating the source video. For example, a reference clock can be selected, and the time on the reference clock increases linearly. The encoder timestamps each video frame based on the time on the reference clock; this timestamp is the PTS. The above is an example, and other methods can also be used, which are not limited here. Alternatively, the content contained in uuid_iso_iec_11578 can be an identifier, such as a QR code or other identifiers. When generating the identifier, it can be converted into a corresponding identifier based on an increasing serial number or sequence number, setting a different identifier for each video frame. This makes the identifier of the SEI information corresponding to each video frame also show an increasing trend, corresponding to the order of the video frames, and can effectively distinguish each video frame. The above SEI information is an example; the specific settings can be set according to the implementation situation, which will not be elaborated here.

[0074] SEI information belongs to the category of bitstream and is additional information added to the video bitstream. SEI information uses bitstream and is added using methods such as extracting the h.264 bitstream from the source video using ffmpeg and adding SEI information using the h264_metadata bitstream filter. The above are just examples, and the specific technical means used to add SEI information are not limited here.

[0075] Furthermore, before adding SEI information to the video frames of the source video, ffmpeg can be used to check if SEI information already exists in the source video. If it already exists, there is no need to add it again, avoiding redundant settings. For example, when using ffmpeg to decapsulate the source video, ffprobe can be used to view various information about the audio and video files, such as the container format, audio / video stream information, and data packet information. By examining the avPacket structure, which stores compressed encoded data in the data packet information, it can be determined whether each video frame in the source video has the corresponding SEI information, avoiding redundant settings.

[0076] Step S202: Obtain SEI information from the source video.

[0077] Before transmitting the source video with added SEI information to the edge computing nodes for transcoding, the source video is decapsulated. For example, using ffmpeg, the source video is decapsulated to obtain the avPacket structure storing the compressed encoded data. The SEI information is set in avPacket. By parsing the avPacket structure, the SEI information of the source video can be obtained.

[0078] Here, only the source video is decapsulated to obtain its SEI information. No decoding or re-encoding is performed, which reduces resource consumption and results in faster processing speed. By simply decapsulating the source video without any other processing, the specific video content remains unchanged, ensuring the source video retains its original content.

[0079] Step S203: Store the SEI information in the cache based on the decoding timestamp of the structure.

[0080] Considering that subsequent transcoding of the source video involves decoding and re-encoding, the SEI information in the transcoded video is modified, becoming inconsistent with the SEI information of the source video, making direct video alignment impossible. Therefore, it is necessary to store the obtained SEI information of the source video beforehand for easy copying and writing to the transcoded video later. Storing the SEI information in a cache allows for direct retrieval and copying after transcoding.

[0081] Furthermore, SEI information corresponds to video frames, with multiple video frames corresponding to multiple SEI information sets. When copying and writing, it's also necessary to determine the specific video frame corresponding to the write operation. During storage, SEI information can be stored based on the DTS (Decoding TimeStamp) in the avPacket. Specifically, DTS tells the player when to decode the data of a particular video frame, corresponding to the video frame. Storing SEI information according to DTS facilitates accurate writing by decoding video frames based on DTS and obtaining the corresponding SEI information during transcoding of the source video.

[0082] Step S204: Transcode the source video, obtain SEI information based on the decoding timestamp of the structure, and copy and write the SEI information to obtain the transcoded video.

[0083] The source video undergoes transcoding, such as decoding it by re-encoding each video frame according to the DTS decoding timestamps. Simultaneously, the SEI information for each video frame is retrieved from the cache based on the DTS decoding timestamps in the structure. This SEI information is then directly copied and written into the transcoded video, resulting in the transcoded video. The SEI information in the transcoded video is a copy of the SEI information from the source video, ensuring consistency between the SEI information in the transcoded video and the SEI information in the source video.

[0084] Step S205: Align the source video and the transcoded video based on the same SEI information in the source video and the transcoded video.

[0085] During video alignment, based on the SEI information contained in the source and transcoded videos, video frames with the same SEI information are aligned to achieve video alignment between the source and transcoded videos. Specifically, the source and transcoded videos are decapsulated, for example using ffmpeg, to obtain the SEI information of both the source and transcoded videos. The SEI information of the source and transcoded videos is then compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information. If the SEI information is an SEI sequence number, then the SEI sequence numbers of each video frame in the source video and the SEI sequence numbers of each video frame in the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video that have the same SEI sequence number. For example, if the SEI sequence number of the first frame of the source video is 1, and the SEI sequence number of the first frame of the transcoded video is also 1, then the first video frame of the source video is the first frame of the source video, and the second video frame of the transcoded video is the first frame of the transcoded video. Alternatively, if the SEI information is a display timestamp, then the display timestamps of the SEI information of each video frame in the source video are compared. The timestamps and the display timestamps of SEI information for each video frame in the transcoded video are used. When the display timestamps in the video frames of the source video and the transcoded video are the same, the first video frame of the source video and the second video frame of the transcoded video with the same display timestamp are identified. Alternatively, the identifier codes of the SEI information for each video frame in the source video and the transcoded video are compared. When the identifier codes in the video frames of the source video and the transcoded video are the same, the first video frame of the source video and the second video frame of the transcoded video with the same identifier codes are identified. Furthermore, the number of first video frames in the source video and the number of second video frames in the transcoded video are the same. For example, after comparison, it is determined that the number of first video frames in the source video is 20, and the number of second video frames in the transcoded video is also 20.

[0086] When there are multiple first and second video frames, for simultaneous playback or storage of the first video frame of the source video and the second video frame of the transcoded video, the first video frames of the source video and the second video frames of the transcoded video with the same SEI information can be sorted according to the playback order. For example, the first video frames of the source video include frame 1, frame 2, frame 3, etc., and the second video frames of the transcoded video include frame 1, frame 2, frame 3, etc. Each is sorted according to its own playback order, such as the order of the PTS display timestamps. A one-to-one correspondence is established between each first and second video frame, such as the first frame of the source video corresponding to the first frame of the transcoded video, the second frame of the source video corresponding to the second frame of the transcoded video, the third frame of the source video corresponding to the third frame of the transcoded video, etc. During playback, the corresponding first and second video frames can be played simultaneously to complete video alignment. The images of the two video frames can be visually observed for quality assessment, etc. During storage, the corresponding first and second video frames are stored according to the correspondence to complete the video alignment of the source and transcoded videos.

[0087] Based on the above processing, taking a 2-second video as an example, compared to decoding operations alone, the processing speed can be increased by approximately 50 times, and the CPU performance usage is reduced from 1764.14% during decoding to 3.14%, significantly reducing resource consumption. In specific implementations, slight variations may occur depending on device performance data. This embodiment significantly improves processing speed and reduces decoding pressure by eliminating the need for decoding during video alignment.

[0088] According to the video alignment method provided in this application, the SEI information of the acquired source video is first stored in a cache. During transcoding, the SEI information of the source video is copied and written into the transcoded video. The SEI information in the final transcoded video is consistent with the SEI information in the source video. By comparing the SEI information in the source video and the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information are determined. The first video frame of the source video and the second video frame of the transcoded video are played and stored simultaneously to complete the video alignment of the source video and the transcoded video. In this process, only the source video and the transcoded video are decapsulated to obtain the SEI information for comparison, without the need to decode the video frames of the source video and the transcoded video, which improves the processing speed and reduces resource consumption.

[0089] Figure 3 A schematic diagram of the structure of a video alignment device provided in an embodiment of this application is shown. Figure 3 As shown, the device includes:

[0090] The acquisition module 310 is adapted to acquire supplementary enhancement information (SEI) information from the source video;

[0091] The copy-write module 320 is suitable for transcoding the source video and copying and writing the SEI information to obtain the transcoded video;

[0092] Alignment module 330 is adapted to perform video alignment between the source video and the transcoded video based on the same SEI information in the source video and the transcoded video.

[0093] Optionally, the acquisition module 310 is further adapted to:

[0094] The source video is decapsulated to obtain a structure that stores compressed encoded data;

[0095] Parse the structure to obtain the SEI information of the source video.

[0096] Optionally, the device further includes:

[0097] The cache module 340 is suitable for storing SEI information in the cache based on the decoding timestamp of the structure.

[0098] Optionally, the copy-write module 320 is further adapted to:

[0099] Transcode the source video;

[0100] Retrieve the corresponding SEI information from the cache based on the decoding timestamp of the structure;

[0101] The SEI information is copied and written into the transcoded video to obtain the transcoded video.

[0102] Optionally, the device further includes:

[0103] Add module 350, which is suitable for adding SEI information to video frames in the source video.

[0104] Optionally, the SEI information can be set in an incremental manner.

[0105] Optionally, the SEI information includes the SEI serial number, display timestamp, and / or identification code.

[0106] Optionally, the alignment module 330 is further adapted to:

[0107] Decapsulate the source video and the transcoded video to obtain the SEI information of the source video and the SEI information of the transcoded video respectively;

[0108] By comparing the SEI information of the source video and the SEI information of the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information are determined.

[0109] Simultaneously play and / or store the first video frame of the source video and the second video frame of the transcoded video to complete the video alignment of the source video and the transcoded video.

[0110] Optionally, the alignment module 330 is further adapted to:

[0111] By comparing the SEI number of the source video and the SEI number of the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI number are determined.

[0112] or,

[0113] Compare the display timestamp of the source video with the display timestamp of the transcoded video to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same display timestamp;

[0114] or,

[0115] By comparing the identifier codes of the source video and the transcoded video, the first video frame of the source video and the second video frame of the transcoded video corresponding to the same identifier code are determined.

[0116] Optionally, the number of the first video frames of the source video and the number of the second video frames of the transcoded video are the same.

[0117] Optionally, the number of first video frames can be multiple; the number of second video frames can also be multiple.

[0118] Alignment module 330 is further adapted to:

[0119] The first video frame of the source video and the second video frame of the transcoded video with the same SEI information are sorted according to the order of video playback and a one-to-one correspondence is established.

[0120] Play and / or store the first and second video frames that have a corresponding relationship simultaneously to complete the video alignment of the source video and the transcoded video.

[0121] The descriptions of the above modules refer to the corresponding descriptions in the method embodiments, and will not be repeated here.

[0122] According to the video alignment device provided in this application, the SEI information of the source video is copied and written into the transcoded video, meaning that the SEI information in the transcoded video and the source video is the same. Based on the identical SEI information in the source video and the transcoded video, video alignment can be performed without decoding the video frames of the source video and the transcoded video, saving resources and significantly improving the processing speed of video alignment.

[0123] This application also provides a non-volatile computer storage medium storing at least one executable instruction that can execute the video alignment method in any of the above method embodiments.

[0124] Figure 4The diagram shows a structural schematic of a computing device according to an embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the computing device.

[0125] like Figure 4 As shown, the computing device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0126] in:

[0127] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0128] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0129] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above-described video alignment method embodiment.

[0130] Specifically, program 410 may include program code that includes computer operation instructions.

[0131] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement this application. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0132] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0133] Specifically, program 410 can be used to cause processor 402 to execute the video alignment method in any of the above method embodiments. The specific implementation of each step in program 410 can be found in the corresponding descriptions of the steps and units in the above video alignment embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0134] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this application.

[0135] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0136] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0137] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0138] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0139] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0140] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A video alignment method, comprising: Obtain the Supplemental Enhancement Information (SEI) from the source video; The SEI information includes the SEI serial number, display timestamp, and / or identification code; The source video is transcoded, and the SEI information is copied and written to obtain the transcoded video; The source video and the transcoded video are decapsulated to obtain the SEI information of the source video and the transcoded video respectively. The SEI information of the source video and the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information. The first video frame of the source video and the second video frame of the transcoded video are played and / or stored simultaneously to complete the video alignment of the source video and the transcoded video. Specifically, the SEI sequence number of the source video and the SEI sequence number of the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI sequence number; or, the display timestamp of the SEI information of the source video and the display timestamp of the SEI information of the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same display timestamp; or, the identifier code of the SEI information of the source video and the identifier code of the SEI information of the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same identifier code.

2. The method according to claim 1, wherein, The step of obtaining SEI information from the source video further includes: The source video is decapsulated to obtain a structure that stores compressed encoded data; The structure is parsed to obtain the SEI information of the source video.

3. The method according to claim 2, wherein, After obtaining the SEI information from the source video, the method further includes: The SEI information is stored in the cache based on the decoding timestamp of the structure.

4. The method according to claim 3, wherein, The step of transcoding the source video and copying and writing the SEI information to obtain the transcoded video further includes: The source video is transcoded. Retrieve the corresponding SEI information from the cache based on the decoding timestamp of the structure; The SEI information is copied and written into the transcoded video to obtain the transcoded video.

5. The method according to claim 1, wherein, The method further includes: SEI information is added to the video frames in the source video.

6. The method according to claim 5, wherein, The SEI information is set in an incremental manner.

7. The method according to claim 1, wherein, The number of the first video frames of the source video is the same as the number of the second video frames of the transcoded video.

8. The method according to any one of claims 1 or 7, wherein, The number of the first video frames is multiple; the number of the second video frames is multiple; The step of simultaneously playing and / or storing the first video frame of the source video and the second video frame of the transcoded video to complete the video alignment of the source video and the transcoded video further includes: The first video frame of the source video and the second video frame of the transcoded video with the same SEI information are sorted according to the order of video playback and a one-to-one correspondence is established. Simultaneously play and / or store the first and second video frames that have a corresponding relationship to complete the video alignment of the source video and the transcoded video.

9. A video alignment device, comprising: The acquisition module is suitable for acquiring the supplementary enhancement information (SEI) from the source video. The SEI information includes the SEI serial number, display timestamp, and / or identification code; The copy-write module is adapted to transcode the source video and copy and write the SEI information to obtain a transcoded video; An alignment module is adapted to decapsulate the source video and the transcoded video, obtain the SEI information of the source video and the transcoded video respectively, compare the SEI information of the source video and the transcoded video, determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI information, and simultaneously play and / or store the first video frame of the source video and the second video frame of the transcoded video to complete the video alignment of the source video and the transcoded video; wherein, comparing the SEI sequence number of the source video and the transcoded video... The SEI sequence number of the transcoded video is used to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same SEI sequence number; or, the display timestamp of the SEI information of the source video and the display timestamp of the SEI information of the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same display timestamp; or, the identifier code of the SEI information of the source video and the identifier code of the SEI information of the transcoded video are compared to determine the first video frame of the source video and the second video frame of the transcoded video corresponding to the same identifier code.

10. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the video alignment method as described in any one of claims 1-8.

11. A computer storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the video alignment method as described in any one of claims 1-8.

12. A computer program product comprising at least one executable instruction that causes a processor to perform an operation corresponding to the video alignment method as described in any one of claims 1-8.

Images