A pixel-level subtitle data moving and superimposing system and method

Abstract

The application discloses a kind of pixel-level subtitle data movement, superimposed system and method, by pre-reading first data before first superposition, so that superposition data can be carried out with reading data synchronization, at the same time, by additionally setting subtitle movement task, so that subtitle moves and superposition data carry out simultaneously, to realize serial operation parallelization, make full use of the time interval of each frame data, repeat this process until superposition is completed.In addition, by WEB interface unit one end connects dynamic generation unit, the other end connects browser, realizes that user directly on WEB network management completes the addition of picture and caption, so that operation is more simple and convenient.YUV superposition is carried out to each frame data of input signal, guarantees the same smooth effect of hardware device, more by making full use of the interval time of each frame image to produce pixel-level caption movement, can achieve smooth movement effect instead of static picture superposition.

Description

Technical Field

[0001] This invention relates to the field of video information processing technology, and in particular to a pixel-level subtitle data movement and overlay system and method. Background Technology

[0002] Subtitle / image overlay technology is widely used in information dissemination, video conferencing, and other occasions where text information can be updated at any time due to its powerful functions.

[0003] Currently, there are two common methods for subtitle overlay: dedicated subtitle generators and encoder-based methods. The dedicated subtitle generator method uses an HDMI / SDI / CVBS signal subtitle overlay unit. This unit employs FPGA image processing and text overlay technology, enabling the overlay of text and images onto high-definition HDMI / SDI or standard-definition CVBS signals, as well as the creation of flowing text. Users can control the displayed text, images, and display mode via RS-232 or a local area network. The subtitle generator inputs a video signal, overlays the image / subtitle, and then outputs the video signal to the subsequent H.264 / H.265 encoder. The encoder performs video compression encoding and outputs a stream containing subtitles. The traditional encoder method involves directly inputting the HDMI / SDI / CVBS signal into the H.264 / H.265 encoder. Specialized software on a PC generates static image and static text data, which is then passed to the encoder. The encoder then overlays the image and subtitle data onto the signal source, completing video compression encoding and outputting a stream containing subtitles.

[0004] However, using a dedicated subtitle generator requires additional equipment to overlay images / subtitles, increasing the complexity of system installation and maintenance, making it more complicated for users, and incurring additional learning and costly expenses. Traditional encoders can only overlay static images and subtitles, failing to achieve smooth overlaying of moving subtitles and dynamic images, and require specialized software installed on a PC. Summary of the Invention

[0005] In view of this, the present invention provides a pixel-level subtitle data movement and overlay system and method, aiming to solve the technical problems of high cost and poor overlay effect caused by the investment of additional equipment and software.

[0006] To address the above technical problems, the present invention provides a pixel-level subtitle data movement and overlay system, comprising: a video input module for receiving source code information of a video source, an overlay module for processing subtitle data, and a compression encoding module for converting the processed subtitle data into a video stream; the overlay module further comprises a dynamic generation unit for overlaying dynamic images and subtitles; the dynamic generation unit is configured to perform a pre-reading task, a subtitle movement task, and an overlay task.

[0007] Optionally, the overlay module further includes a YUV data unit and a YUV data overlay unit. The YUV data unit is used to convert the source code information received by the video input module into YUV data information, and the YUV data overlay unit is used to overlay the data generated by the YUV data unit and the dynamic generation unit.

[0008] Optionally, the overlay module further includes a WEB interface unit, one end of which is connected to the dynamic generation unit and the other end of which is connected to a browser.

[0009] Optionally, the browser and the dynamic generation unit interact with each other through the WEB interface unit, and the browser is also used to process subtitle data for the dynamic generation unit.

[0010] Optionally, the pre-read task is configured to read the first data before data overlay and synchronously cache the second data while the overlay task is in progress. The second data is obtained by splicing two third data of the same width.

[0011] Optionally, the subtitle movement task is configured to simultaneously move the subtitles on the first data when the overlay task is in progress.

[0012] Accordingly, the present invention also provides a pixel-level subtitle data movement and overlay method, comprising the following steps:

[0013] S1. Obtain video source code data and data to be overlaid, and preprocess the video source code data;

[0014] S2. Pre-read the first data in the data to be superimposed;

[0015] S3. While superimposing the first data with the preprocessed data, simultaneously cache the second data in the data to be superimposed;

[0016] S4. While superimposing the first data with the preprocessed data, simultaneously execute the subtitle movement task;

[0017] S5. Repeat steps S2-S4 until all video source data and data to be superimposed are superimposed. Then compress and encode all the superimposed data to form a video stream output.

[0018] Optionally, the preprocessing of the video source data includes:

[0019] The video source data is converted to YUV to obtain YUV data for data overlay.

[0020] Optionally, steps S2 and S3 can be performed simultaneously.

[0021] This invention provides a pixel-level subtitle data movement and overlay system and method. By pre-reading the first data before the first overlay, the overlay data can be processed synchronously with the read data. Simultaneously, by adding a subtitle movement task, subtitle movement and overlay data are carried out concurrently, achieving parallelization of serial operations and fully utilizing the time interval between each frame of data. This process is repeated until overlay is complete. Furthermore, a web interface unit connects to a dynamic generation unit on one end and a browser on the other, allowing users to directly add images and subtitles on the web interface, making the operation simpler and more convenient. This invention achieves the same functionality without requiring additional hardware, reducing costs. Moreover, it enables the overlay of multiple video streams on a single encoder, saving on the investment in multiple devices. Furthermore, YUV overlay is performed on each frame of the input signal to ensure consistent smoothness across the hardware. By fully utilizing the time interval between each frame of image to generate pixel-level subtitle movement, a smooth movement effect is achieved, rather than static image overlay. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a pixel-level subtitle data movement and overlay system provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram illustrating the steps of a pixel-level subtitle data movement and overlay method provided in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the task executed by the dynamic generation unit according to an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention provides a pixel-level subtitle data movement and overlay system mounted on an encoder. The specific principle of the invention lies in the fact that data overlay is directly implemented within the encoder without the need for additional equipment. Furthermore, YUV overlay is performed on each frame of the input signal, achieving the same smooth effect as hardware devices. Simultaneously, the system fully utilizes the interval between each frame to generate pixel-level subtitle movement, resulting in a smooth movement effect. Moreover, by utilizing the browser's H5 functionality, the need for dedicated PC software is eliminated, simplifying the operation and improving the user experience.

[0028] Reference Figure 1 This is a schematic diagram of a pixel-level subtitle data movement and overlay system provided in an embodiment of the present invention, including: a video input module for receiving source code information of a video source, an overlay module for processing subtitle data, and a compression encoding module for converting the processed subtitle data into a video stream; the overlay module further includes a dynamic generation unit for overlaying dynamic images and subtitles; the dynamic generation unit is configured to perform a pre-reading task, a subtitle movement task, and an overlay task.

[0029] Furthermore, the overlay module also includes a YUV data unit and a YUV data overlay unit. The YUV data unit is used to convert the source code information received by the video input module into YUV data information, and the YUV data overlay unit is used to overlay the data generated by the YUV data unit and the dynamic generation unit.

[0030] Furthermore, the overlay module also includes a WEB interface unit, one end of which is connected to the dynamic generation unit and the other end is connected to a browser.

[0031] Furthermore, the browser and the dynamic generation unit interact with each other through the WEB interface unit, and the browser is also used to process subtitle data for the dynamic generation unit. Specifically, users can add, delete, and modify subtitles through the browser's H5 function, and can also upload videos or images.

[0032] Furthermore, the pre-read task is configured to cache the first data before data overlay and synchronously read the second data during the overlay task. The second data is obtained by splicing two third data of the same width.

[0033] Furthermore, the subtitle movement task is configured to simultaneously move the subtitles on the first data during the overlay task.

[0034] Understandably, the video source sends the source video to the video input module, which first converts the source video from YUV to YUV data. Then, through the WEB interface unit, it connects to the browser and the dynamic generation unit respectively, allowing the user to exchange information with the dynamic generation unit through the browser to submit the data to be overlaid. Then, the overlay module executes the overlay task on the YUV data and the data to be overlaid. Finally, the overlaid data is processed by the compression encoding module to obtain the processed video stream for output.

[0035] Furthermore, the dynamic generation unit is configured to execute pre-reading, subtitle movement, and overlay tasks. These tasks are used to perform image overlay and subtitle movement processing on the data to be overlaid. Before the overlay task begins, the pre-reading task caches a portion of data as initial data, ensuring that the subtitle movement and overlay tasks are performed on-demand data after the overlay task starts. By moving subtitles based on the subtitle movement task and overlaying images based on the overlay task, subtitle movement and image processing can be performed simultaneously, achieving parallelization of serial operations.

[0036] This invention provides a pixel-level subtitle data movement and overlay system. By integrating the system into the encoder and configuring it based on a dynamic generation unit, it not only achieves the same functionality without requiring additional hardware, reducing costs, but also enables the overlay of multiple video streams on a single encoder, saving on the investment in multiple devices. Furthermore, it performs YUV overlay on each frame of the input signal, ensuring consistent smoothness across the hardware. By fully utilizing the interval between each frame to generate pixel-level subtitle movement, it achieves a smooth movement effect rather than static image overlay. In addition, the generation of image / subtitle data utilizes the browser's H5 functionality, eliminating the need for dedicated PC software, making operation simpler and more convenient, and significantly reducing the user's learning curve.

[0037] Accordingly, the present invention also provides a pixel-level method for moving and overlaying subtitle data, referring to... Figure 2 As shown, the following steps are specifically executed, wherein the content of steps S2-S4 is combined with Figure 3 Further understanding:

[0038] S1. Obtain video source code data and data to be overlaid, and preprocess the video source code data;

[0039] First, obtain the video source data from the video source HDMI / SDI / CVBS, obtain the data to be overlaid from the browser, and process the obtained video source data through YUV conversion to obtain the processed YUV data.

[0040] S2. Pre-read the first data in the data to be superimposed;

[0041] It's important to note that there's a latency issue when storing data from Flash to memory. Moving captions / animated images are stored in Flash, and due to the large data volume, they cannot all be loaded into memory at once. They need to be loaded dynamically in multiple parts. This loading process inevitably involves a latency, and if the latency exceeds the interval between video frames, it will cause stuttering. Therefore, before the data overlay task begins, a portion of the data to be overlaid is pre-fetched and used as the first data. Specifically, two images of the same width are read and stitched together to form an image twice the width. After each frame of overlay is completed, the overlay / caption movement task shifts the stitched image by 2 pixels, and the next overlay uses the shifted image data.

[0042] Specifically, in actual use, to achieve the overlay of two 1080P 60fps videos, the interval between each frame is 1000ms / (60×2), which is 8.333ms. In other words, theoretically, the operations that need to be performed within 8.333ms are: preparing the data to be overlaid in the current frame and overlaying the prepared data into the video. Based on the latency during memory loading, the data reading latency already exceeds 8.333ms. Therefore, before the data overlay task begins, a portion of the data to be overlaid is pre-fetched. Each time, two 1920-pixel wide images are read, which are then stitched together to form a 3840-pixel wide image. The overlay / moving caption task moves the 3840-pixel image by 2 pixels after each frame overlay. The next overlay uses the moved image data. Thus, the time from the start of overlay to completion can be calculated as: (1920 / 2) × (1000 / 60) = 1600ms. The pre-fetch task only needs to ensure that the next image to be used is read into memory within 1600ms. Even if there is a latency in data reading, the latency and the total data reading time will not exceed 1600ms, thus solving the data reading latency problem and preventing stuttering caused by data reading.

[0043] S3. While superimposing the first data with the preprocessed data, simultaneously cache the second data in the data to be superimposed;

[0044] The first data is pre-read before overlaying. Therefore, when performing the overlay task, the available data is the first data. There is no situation where there is no data to read when performing the overlay task, and there is no need to wait for the data to be read before overlaying. At this time, when performing the overlay task on the first data, the pre-read task will generate a new buffer to store the next data, that is, synchronously cache the second data. When the overlay task completes the overlay task on the first data, the second data is cached. The overlay task can continue to cache the second data, and the pre-read task continues to read the next data until all data is read.

[0045] S4. While superimposing the first data with the preprocessed data, simultaneously execute the subtitle movement task;

[0046] It should be noted that each frame of video data corresponds to one subtitle movement, and the time taken for each movement is equal to the subtitle data pixel movement time plus the overlay time. Since the overlay time is fixed and cannot be optimized, the optimization method for the subtitle data pixel movement time is to use a dynamically generated unit configured with subtitle movement tasks. When the overlay task is executed, the subtitle movement task is executed synchronously. The principle is the same as the pre-read task mentioned above. Since the first data already exists before the overlay task, the subtitle movement task can be performed synchronously on the first data. This achieves the simultaneous movement of subtitles and overlays data, making parallelization of serial operations and making full use of the time interval of each frame.

[0047] S5. Repeat steps S2-S4 until all video source data and data to be superimposed are superimposed. Then compress and encode all the superimposed data to form a video stream output.

[0048] Repeat steps S2-S4 until all data is superimposed. At this time, the subtitle movement task is also completed. All superimposed data is processed by the compression encoding module to obtain the video stream, and then the video stream is output.

[0049] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0050] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be considered as limitations on the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. For those skilled in the art, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A system for overlaying dynamic images and subtitles from multiple video streams, characterized in that, include: A video input module for receiving source code information of video source, an overlay module for processing subtitle data, and a compression encoding module for converting processed subtitle data into a video stream. The overlay module also includes a dynamic generation unit for overlaying dynamic images and subtitles; The dynamic generation unit is configured to perform pre-reading tasks, subtitle movement tasks, and overlay tasks; wherein... The pre-read task is configured to read two images of the same width before data overlay, and then stitch these two images of the same width together to form an image with twice the width. During the overlay task, two images of the same width are read synchronously. The subtitle movement task is configured to simultaneously move the subtitles on the spliced ​​image, which is twice the width of the image, during the overlay task.

2. The multi-channel video dynamic image and subtitle overlay system according to claim 1, characterized in that, The overlay module further includes a YUV data unit and a YUV data overlay unit. The YUV data unit is used to convert the source code information received by the video input module into YUV data information, and the YUV data overlay unit is used to overlay the data generated by the YUV data unit and the dynamic generation unit.

3. The dynamic image and subtitle overlay system for multi-channel video according to claim 1, characterized in that, The overlay module also includes a WEB interface unit, one end of which is connected to the dynamic generation unit and the other end is connected to a browser.

4. A multi-channel video dynamic image and subtitle overlay system according to claim 3, characterized in that, The browser and the dynamic generation unit interact with each other through the WEB interface unit, and the browser is also used to process subtitle data for the dynamic generation unit.

Images