Method, system and terminal for adding extra annotation information during video playing

Abstract

The invention discloses a method, a system and a terminal for adding extra annotation information during video playing, and the method comprises the steps: constructing a video playing assembly which comprises a first display layer and a second display layer, the first display layer is used for video playing, and the second display layer is used for annotation drawing; acquiring a video source and annotation information; according to the video source, displaying a video frame through the first display layer, and according to the annotation information, drawing an annotation graph through the second display layer; and performing image synthesis according to the video frame and the annotation graph to obtain a synthesized image, and displaying the synthesized image. According to the method and the device, the video playing component supporting two-layer display is initialized, the video playing loop and the label drawing loop are performed to display the video and the standard in the corresponding layer, and the two layers of contents are subjected to image merging and output to the screen, so that the efficiency of adding extra label information during video playing of the OH system is improved.

Description

Technical Field

[0001] The present application relates to the field of video playback technology, and in particular to a method, system and terminal for adding additional annotation information during video playback. Background Art

[0002] During video playback, if you need to overlay graphics on the video based on the annotation information, the common practice is to use a decoder to decode the video into image frames, then overlay the annotated image on each frame in the program, and finally display the synthesized image on the window. The video source and the annotation source are separate, and it is necessary to overlay the graphics drawn according to the annotation information on the video while it is playing. In a common situation under Windows system, after the decoder produces a frame of image, the program overlays the annotated image on it, and then displays the image on the window. The special thing about the OH (OpenHarmony) system is that neither the multimedia nor the driver touches the image after the video is decoded, and the bottom layer directly sends the graphics for display, which makes the OH system less efficient in adding additional annotation information when playing videos.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of the present application is to provide a method, system and terminal for adding additional annotation information during video playback, aiming to solve the problem in the prior art that adding additional annotation information to video playback requires processing the image after video decoding, while the multimedia and driver of the OH system do not touch the image after video encoding, resulting in low efficiency in adding additional annotation information during video playback of the OH system.

[0005] A first aspect of an embodiment of the present application provides a method for adding additional annotation information during video playback, and the method for adding additional annotation information during video playback includes the following steps: constructing a video playback component, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing; obtaining a video source and annotation information; displaying a video frame through the first display layer according to the video source, and drawing an annotation graphic through the second display layer according to the annotation information; performing image synthesis according to the video frame and the annotation graphic to obtain a synthesized image, and displaying the synthesized image.

[0006] Optionally, in one embodiment of the present application, the construction of the video playback component specifically includes: creating a bottom display layer and a top display layer; setting the bottom attributes of the bottom display layer and the top attributes of the top display layer; and display-associating the bottom display layer with the top display layer according to the bottom attributes and the top attributes, using the associated bottom display layer as the first display layer and the associated top display layer as the second display layer.

[0007] Optionally, in one embodiment of the present application, the obtaining of the video source and annotation information specifically includes: responding to a control operation on a video file, reading the video file corresponding to the control operation to obtain the video source; obtaining the annotation source, and reading the annotation information from the annotation source.

[0008] Optionally, in an embodiment of the present application, displaying the video frame through the first display layer according to the video source specifically includes: when the video source is played, parsing the video source to obtain parsed data;

[0009] The parsed data is input into a decoder to obtain a decoded video frame; and the video frame is synchronously displayed on the first display layer.

[0010] Optionally, in one embodiment of the present application, drawing the annotation graphics through the second display layer according to the annotation information specifically includes: obtaining the timestamp corresponding to the video frame, and determining the annotation data corresponding to the timestamp in the video source; processing the annotation data to obtain processed annotation data to adapt to the second display layer; and drawing the annotation graphics on the second display layer according to the processed annotation data.

[0011] Optionally, in one embodiment of the present application, the image synthesis is performed according to the video frame and the annotated graphic to obtain a synthesized image, and the synthesized image is displayed, which specifically includes: obtaining a video layer corresponding to the video frame and an annotation layer corresponding to the annotated graphic; using an image processing library or API to synthesize the video layer and the annotation layer to obtain a synthesized image, and outputting the synthesized image for display.

[0012] Optionally, in one embodiment of the present application, the image synthesis is performed according to the video frame and the annotated graphics to obtain a synthesized image, and then further includes: when the video source is played to the end, releasing the resources corresponding to the video source and the annotated source respectively, and recording the performance data of resource management.

[0013] A second aspect of the embodiment of the present application further provides a system for adding additional annotation information during video playback, wherein the system for adding additional annotation information during video playback includes:

[0014] A component initialization module, used to construct a video playback component, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing;

[0015] File information acquisition module, used to obtain video source and annotation information;

[0016] A video playback and annotation drawing module, configured to display video frames through the first display layer according to the video source, and to draw annotation graphics through the second display layer according to the annotation information;

[0017] The image synthesis module is used to perform image synthesis according to the video frame and the annotated graphic to obtain a synthesized image, and display the synthesized image.

[0018] The third aspect of an embodiment of the present application also provides a terminal, wherein the terminal includes: a memory, a processor, and a program for adding additional annotation information during video playback that is stored in the memory and can be run on the processor, and when the program for adding additional annotation information during video playback is executed by the processor, the steps of the method for adding additional annotation information during video playback as described above are implemented.

[0019] The fourth aspect of the embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program for adding additional annotation information during video playback, and when the program for adding additional annotation information during video playback is executed by a processor, the steps of the method for adding additional annotation information during video playback as described above are implemented.

[0020] Beneficial effects: The present application provides a method, system and terminal for adding additional annotation information during video playback. In the method, a video playback component is initialized to support two-layer display, a video playback loop and an annotation drawing loop are performed to display the video and standard on the corresponding layers, and the two layers of content are merged and output to the screen as images, thereby improving the efficiency of adding additional annotation information during video playback of the OH system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a flowchart of a preferred embodiment of the method for adding additional annotation information during video playback of the present application;

[0023] Figure 2 It is a structural schematic diagram of a preferred embodiment of the system for adding additional annotation information during video playback of the present application;

[0024] Figure 3 This is a schematic diagram of the structure of a preferred embodiment of the terminal of the present application.

[0025] Description of reference numerals:

[0026] 10. System for adding additional annotation information during video playback; 100. Component initialization module; 200. File information acquisition module; 300. Video playback and annotation drawing module; 400. Image synthesis module. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and effect of the present application clearer and more specific, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. The described embodiments are only possible technical implementations of the present application, not all possible implementations. Based on the embodiments in the present application, those skilled in the art can completely combine the embodiments of the present application to obtain other embodiments without creative work, and these embodiments are also within the scope of protection of the present application.

[0028] In the related technology, a decoder is used to decode the video file into a series of frames, and annotation information is drawn on each frame. The drawn annotation frame is synthesized with the original video frame, and the synthesized frame is displayed on the window. That is, the image gives the data to the program, the program continues to draw things, and the program directly returns to the image data. In the OH system, since the multimedia and the driver do not touch the image after the video is decoded, the bottom layer directly sends the graphics to the display (it cannot receive the data after the hardware decoding).

[0029] The following describes the method, system and terminal for adding additional annotation information during video playback according to an embodiment of the present application with reference to the accompanying drawings. In view of the problem that the video playback in the above-mentioned related art needs to process the image after video decoding, and the multimedia and driver of the OH system do not touch the image after video encoding, resulting in low efficiency of adding additional annotation information during video playback of the OH system, the present application provides a method for adding additional annotation information during video playback, in which a video playback component is initialized to support two-layer display, a video playback loop and an annotation drawing loop are performed to display the video and standard on the corresponding layer, and the two layers of content are merged and output to the screen, thereby improving the efficiency of adding additional annotation information during video playback of the OH system. Thus, the technical problem that the video playback in the related art needs to process the image after video decoding, and the multimedia and driver of the OH system do not touch the image after video encoding, resulting in low efficiency of adding additional annotation information during video playback of the OH system is solved.

[0030] In an embodiment of the present application, two display layers are created on the video component, the bottom layer is used for normal video playback, and the upper layer is used for annotation drawing. When the image synthesis service is processed, it is as if this component is two windows put together, so that graphics drawn according to the annotation information can be superimposed on it while the video is playing.

[0031] The embodiments of the present application have the following advantages:

[0032] (1) Dual-layer display structure: By creating two display layers on the video component, the video content and annotation content are separated, making the addition and modification of annotations more flexible.

[0033] (2) No contact with hardware-decoded data: Multimedia and drivers do not directly contact the images after video decoding, but send the graphics directly to the display through the bottom layer, which reduces the consumption of system resources and improves efficiency.

[0034] (3) Real-time annotation and video synchronization: The annotation drawing loop is synchronized with the video playback loop, ensuring that the annotation information can correspond to the video content in real time, enhancing the accuracy and real-time performance of the annotation.

[0035] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0036] The method for adding additional annotation information during video playback described in the preferred embodiment of the present application is as follows: Figure 1 As shown, the method for adding additional annotation information during video playback includes the following steps:

[0037] In step S101, a video playback component is constructed, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing.

[0038] In a possible implementation, a video playback component supporting two layers of display is initialized, one layer for video playback and one layer for annotation. A bottom display layer and a top display layer are created; bottom attributes of the bottom display layer and top attributes of the top display layer are set; according to the bottom attributes and the top attributes, the bottom display layer is displayed associated with the top display layer, and the associated bottom display layer is used as the first display layer, and the associated top display layer is used as the second display layer.

[0039] Specifically, select a video playback component: select a video playback component that supports multi-layer display, such as FFmpeg or DirectShow; create display layers: create two display layers on the video playback component, the bottom layer is used for video playback, and the top layer is used for annotation drawing. The two layers can be regarded as independent windows and will not interfere with each other; set the bottom layer properties: set the bottom layer to ensure that it can play the video normally, including configuring the decoder, setting the pixel format, etc., to ensure that the video can be played smoothly; set the top layer properties: set the top layer to ensure that it can overlay annotation information on the video, including setting a transparent background, adjusting the coordinate system, etc., so that the annotation information can be accurately covered on the video; associate two layers of display: ensure that the bottom layer and the top layer can work together, adjust the window properties, so that the top layer can move with the bottom layer and always remain displayed on the top layer. Event processing: if necessary, you can add an event processing mechanism to the top layer window to respond to user interaction. For example, when the user clicks or drags the top layer window, the annotation information can be updated in real time and fed back to the user; resource management: allocate appropriate memory and computing resources to each display layer to avoid performance bottlenecks in subsequent operations.

[0040] That is to say, the embodiments of the present application can lay a solid foundation for subsequent video playback and annotation work by reasonably setting and using a video playback component that supports multi-layer display.

[0041] In step S102, video source and annotation information are obtained.

[0042] In a possible implementation, the video source is prepared, the video file is loaded and the decoder is set; the annotation source is prepared, the annotation information is loaded and the drawing module is initialized. In response to the control operation of the video file, the video file corresponding to the control operation is read to obtain the video source; the annotation source is obtained, and the annotation information is read from the annotation source.

[0043] Specifically, select the video source: specify the video file or stream to be played, which can be a local file, a network stream, or other types of video input; load the video source: read the video file or stream and load its content into the memory. For local files, it involves file I / O operations, and for network streams, network communication and data reception are required; set the decoder: select the appropriate decoder according to the encoding format of the video file. If the video is H.264 encoded, you need to use a decoder that supports H.264; configure the decoder parameters: make necessary configurations for the decoder, such as setting parameters such as resolution and frame rate. The parameters should match the properties of the video source to ensure that the video can be correctly decoded and played; initialize the decoder: start the decoder and prepare to receive video frames. The decoder initialization sequence is required, as well as the necessary connection with the video source; cache strategy: in order to improve playback efficiency, cache the video data before decoding; resource management: when loading the video source, consider the use of memory and CPU resources; error handling: in the process of loading the video source, properly handle errors such as file non-existence and network interruption to ensure the stability and reliability of the system.

[0044] Specify the source of annotation data: determine the source of annotation information, including local files, databases, network services or real-time data streams, etc.; Load annotation data: read annotation information from the specified data source, involving file I / O operations, database queries, network requests, etc.; Parse annotation data: parse the acquired annotation data (which may be in XML, JSON, etc. format) into internally usable structured data, such as objects, lists or dictionaries, etc.; Initialize the drawing module: select and configure the graphics library or module used to draw annotations, such as OpenGL, DirectX, Cairo or HTML5 Canvas, etc.; Set drawing parameters: set drawing parameters such as color, font, line width, transparency, etc. according to the characteristics of annotation data and video frames; Create drawing buffer: In order to draw annotations efficiently, it may be necessary to create a drawing buffer, which helps reduce the performance overhead of drawing directly to the screen; Resource management: When initializing the drawing module, it is necessary to reasonably allocate and manage memory, GPU resources, etc. to optimize performance and stability; Error handling: In the process of loading and parsing annotation data, it is necessary to handle possible errors and exceptions, such as format errors, data corruption, etc.; User interaction: If the annotation system allows user interaction, such as editing annotation content, it is necessary to initialize the corresponding interaction interface and event handling logic.

[0045] The embodiment of the present application closely collaborates with the video playback and image synthesis modules through the loading and parsing of the annotation data to ensure the accuracy and real-time nature of the annotation information.

[0046] In step S103, according to the video source, the video frame is displayed through the first display layer, and according to the annotation information, the annotation graphic is drawn through the second display layer.

[0047] In a possible implementation, the video playback loop: decodes the video and plays it on the bottom display layer; when the video source is played, parses the video source to obtain parsed data; inputs the parsed data into a decoder to obtain a decoded video frame; and synchronously displays the video frame to the first display layer.

[0048] Specifically, before the video playback loop starts, initialize the decoder and set its parameters to match the encoding format of the video source; prepare the buffer: prepare an appropriate buffer to store the decoded video frames; start the decoding loop: enter a loop in which data is continuously read from the video source and fed into the decoder; read video data: read a certain amount of data from the video source, which can be a block of a file, a data packet of a network stream, etc.; decode the video frame: send the read data to the decoder, and the decoder will output the decoded video frame; process the decoding result: check the status of the decoding result, such as whether there is a decoding error, and if there is an error, corresponding error handling is required; synchronize the display: synchronize the decoded video frame to the underlying display layer, which involves the graphics library to ensure that the video frame is displayed at the correct position and time; refresh display: instruct the display device to refresh the display layer to show the latest video frame; resource management: during playback, continuously monitor resource usage, such as CPU, memory and GPU usage, to avoid performance bottlenecks; user interaction processing: if the system allows users to interact with the video (such as pause, fast forward, etc.), these user instructions need to be processed in a loop; performance monitoring: real-time monitoring of video playback performance, including frame rate, latency, etc., to ensure smooth playback; error recovery: during playback, if a decoding error or playback problem occurs, a mechanism is needed to recover and restart the playback loop; loop control: decide when to end the playback loop based on the length of the video source and the user's operation.

[0049] That is to say, the embodiment of the present application uses a video playback loop to continuously display the decoded video frames on the screen, forming a smooth video playback effect, and through precise time control and efficient data processing, to ensure the continuity and stability of video playback.

[0050] In a possible implementation, the annotation drawing loop synchronously draws the annotation information on the top display layer, obtains the timestamp corresponding to the video frame, determines the annotation data corresponding to the timestamp in the video source, processes the annotation data to obtain processed annotation data to adapt to the second display layer, and draws the annotation graphics on the second display layer according to the processed annotation data.

[0051] Specifically, initialize the drawing environment: before the annotation drawing loop starts, initialize the drawing module and set drawing parameters such as color, font, line width, etc.; prepare the drawing buffer: create a drawing buffer, which helps reduce the performance overhead of drawing directly to the screen; start the drawing loop: independently of the video playback loop, start a drawing loop to continuously update and display the annotation information; obtain the current video frame information: obtain the information of the currently displayed video frame from the video playback loop, such as frame number, timestamp, etc., to ensure that the annotation is synchronized with the video frame; read the annotation data: according to the information of the current video frame, obtain the corresponding annotation information from the annotation data source; process the annotation data: process the acquired annotation data, such as converting coordinates, adjusting size, applying style, etc., to adapt to the current display layer; draw Annotation: Use the drawing module to draw the processed annotation information onto the top display layer, which involves calling the graphics library; Refresh display: Instruct the display device to refresh the top display layer to display the latest annotation information; Resource management: During the drawing process, continuously monitor resource usage, such as CPU and GPU usage, to avoid performance bottlenecks; User interaction processing: If the system allows users to interact with annotations (such as editing, deleting, etc.), these user instructions need to be processed in a loop; Performance monitoring: Real-time monitoring of the performance of annotation drawing, including frame rate, latency, etc., to ensure smooth and accurate annotation; Error recovery: During the drawing process, if an error or exception is encountered, a mechanism is needed to recover and restart the drawing loop; Loop control: Decide when to end the drawing loop based on the status of video playback and user operations.

[0052] That is to say, the embodiment of the present application ensures that the annotation information can be updated and displayed synchronously with the video content by running the annotation drawing loop in parallel with the video playback loop, precise time control and efficient data processing to ensure the accuracy and real-time nature of the annotation.

[0053] In step S104, an image is synthesized according to the video frame and the annotated graphic to obtain a synthesized image, and the synthesized image is displayed.

[0054] In a possible implementation, a video layer corresponding to the video frame and an annotation layer corresponding to the annotation graphic are obtained; an image processing library or API is used to synthesize the video layer and the annotation layer to obtain a synthesized image, and the synthesized image is output for display.

[0055] Specifically, initialize the synthesis module, which is used to process image synthesis and merge the contents of different layers into the final image; obtain layer frames: obtain the bottom-level video frames and top-level annotation frames from the video playback loop and annotation drawing loop respectively; adjust layer properties: ensure that the size, resolution and pixel format of the two layers are consistent for easy synthesis; apply transparency and blending mode: if the top-level annotation requires a transparent background or special blending effects, apply these settings to ensure that the annotation can be correctly superimposed on the video; synthesize layers: use image processing libraries or APIs to merge two layers into a complete image, involving pixel-level operations such as Alpha blending; optimize image quality: as needed, the quality of the synthesized image can be optimized, such as adjusting contrast, brightness, sharpening etc.; Cache synthesis results: In order to improve performance, cache the synthesized image to avoid re-synthesis operation every time it is displayed; Display synthesized image: Output the synthesized image to a display device, such as a monitor or projector; Resource management: During the synthesis process, continuously monitor resource usage, such as CPU and GPU usage, to avoid performance bottlenecks; Error handling: If an error is encountered during the synthesis process, such as insufficient resources or image format mismatch, error handling is required; Performance monitoring: Real-time monitoring of image synthesis performance, including frame rate, latency, etc., to ensure the smoothness and efficiency of the synthesis process; User interaction response: If the system allows users to interact with the synthesized image, such as zooming in, zooming out, rotating, etc., it is necessary to respond to these interactive operations and update the synthesized image in real time.

[0056] That is to say, the embodiment of the present application forms the final display effect by seamlessly combining the video content and the annotation information.

[0057] In a possible implementation, when the video source is played to the end, the resources corresponding to the video source and the annotation source are released, and the performance data of resource management is recorded.

[0058] Specifically, resources are released after the video playback ends. Monitor resource usage: During video playback, the system needs to continuously monitor resource usage, including memory, CPU, GPU usage, etc.; Detect video end: When the video reaches the end, the resource release process is triggered after detection; Stop video source: Once the video end is detected, first stop the video source reading and decoding operations to ensure that no new data is generated; Stop video playback loop: Stop the video playback loop to ensure that video frames are no longer rendered to the screen; Stop annotation drawing loop: If the system includes annotation function, the annotation drawing loop also needs to be stopped; Release decoder resources: The decoder is a key component for processing video data. Once it is no longer needed, the resources it occupies should be released; Release graphics resources: During video playback and Any graphics resources used in the annotation drawing process, such as textures, buffers, etc., should be released; release video source resources: close the video file or network stream and release all resources related to it; release annotation data resources: if the annotation data occupies memory or file handles, these resources should also be released; reset the user interface: after the video playback is over, the user interface may need to be updated to reflect the status that the playback has ended; resource cleanup confirmation: confirm that all related resources have been released correctly to avoid memory leaks and other resource leaks; status update: update the internal state of the video playback system to prepare for the next playback; performance logging: record performance data of resource management, such as the time required for resource release, which will help future performance optimization.

[0059] It is worth noting that the difference between the embodiment of the present application and the prior art solution is that:

[0060] Dual display layers: This application creates two display layers inside the video playback component instead of operating on a single frame buffer;

[0061] Separate processing: Video decoding and annotation drawing are independent processing processes, reducing mutual interference;

[0062] System resource utilization: Since multimedia and drivers do not touch the decoded video images, the consumption of system resources is reduced;

[0063] Real-time: Annotation drawing and video playback are performed synchronously, ensuring the real-time nature of annotation;

[0064] User interactivity: Provides better user interaction capabilities.

[0065] Next, a system for adding additional annotation information during video playback according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0066] Figure 2 It is a block diagram of a system for adding additional annotation information during video playback according to an embodiment of the present application.

[0067] like Figure 2 As shown, the system 10 for adding additional annotation information during video playback includes: a component initialization module 100 , a file information acquisition module 200 , a video playback and annotation drawing module 300 and an image synthesis module 400 .

[0068] Specifically, the component initialization module 100 is used to construct a video playback component, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing;

[0069] The file information acquisition module 200 is used to acquire the video source and annotation information;

[0070] The video playing and annotation drawing module 300 is used to display the video frame through the first display layer according to the video source, and to draw the annotation graphics through the second display layer according to the annotation information;

[0071] The image synthesis module 400 is used to perform image synthesis according to the video frame and the annotated graphic to obtain a synthesized image, and display the synthesized image.

[0072] Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0073] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0074] When the processor 502 executes the program, the method for adding additional annotation information during video playback provided in the above embodiment is implemented.

[0075] Furthermore, the terminal further includes:

[0076] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0077] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0078] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0079] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EIS) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0080] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0081] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0082] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for adding additional annotation information during video playback as described above is implemented.

[0083] One embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the present application Figure 1 A method for adding additional annotation information during video playback provided in any of the corresponding embodiments.

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0086] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0087] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0088] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0089] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0090] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0091] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0092] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for adding additional annotation information during video playback, characterized in that: The method for adding additional annotation information during video playback includes: Constructing a video playback component, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing; Get video source and annotation information; According to the video source, display the video frame through the first display layer, and according to the annotation information, draw the annotation graphic through the second display layer; An image is synthesized according to the video frame and the annotated graphic to obtain a synthesized image, and the synthesized image is displayed.

2. The method for adding additional annotation information during video playback according to claim 1, characterized in that: The constructing of the video playback component specifically includes: Create a bottom display layer and a top display layer; Setting the bottom layer attributes of the bottom display layer and the top layer attributes of the top display layer; According to the bottom layer attributes and the top layer attributes, the bottom layer display layer is display-associated with the top layer display layer, and the associated bottom layer display layer is used as the first display layer, and the associated top layer display layer is used as the second display layer.

3. The method for adding additional annotation information during video playback according to claim 1, characterized in that: The obtaining of video source and annotation information specifically includes: In response to a control operation on a video file, the video file corresponding to the control operation is read to obtain a video source; Obtain a marking source, and read marking information from the marking source.

4. The method for adding additional annotation information during video playback according to claim 1 or 2, characterized in that: The displaying the video frame through the first display layer according to the video source specifically includes: When the video source is played, the video source is parsed to obtain parsed data; Inputting the parsed data into a decoder to obtain a decoded video frame; The video frame is synchronously displayed on the first display layer.

5. The method for adding additional annotation information during video playback according to claim 4, characterized in that: Drawing the annotation graphic by the second display layer according to the annotation information specifically includes: Obtaining a timestamp corresponding to the video frame, and determining the annotation data corresponding to the timestamp in the video source; Processing the annotation data to obtain processed annotation data to adapt to the second display layer; Drawing a marked graphic on the second display layer according to the processed marked data.

6. The method for adding additional annotation information during video playback according to claim 5, characterized in that: The step of synthesizing the image according to the video frame and the annotated graphic to obtain a synthesized image and displaying the synthesized image specifically includes: Obtaining a video layer corresponding to the video frame and an annotation layer corresponding to the annotation graphic; The video layer and the annotation layer are synthesized by using an image processing library or API to obtain a synthesized image, and the synthesized image is output for display.

7. The method for adding additional annotation information during video playback according to claim 3, characterized in that: The step of performing image synthesis according to the video frame and the annotated graphic to obtain a synthesized image further includes: When the video source is played to the end, the resources corresponding to the video source and the annotation source are released, and the performance data of resource management is recorded.

8. A system for adding additional annotation information during video playback, characterized in that: The system for adding additional annotation information during video playback includes: A component initialization module, used to construct a video playback component, wherein the video playback component includes a first display layer and a second display layer, the first display layer is used for video playback, and the second display layer is used for annotation drawing; File information acquisition module, used to obtain video source and annotation information; A video playback and annotation drawing module, configured to display video frames through the first display layer according to the video source, and to draw annotation graphics through the second display layer according to the annotation information; The image synthesis module is used to perform image synthesis according to the video frame and the annotated graphic to obtain a synthesized image, and display the synthesized image.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a program for adding additional annotation information during video playback that is stored in the memory and can be run on the processor. When the program for adding additional annotation information during video playback is executed by the processor, the steps of the method for adding additional annotation information during video playback as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for adding additional annotation information during video playback, and when the program for adding additional annotation information during video playback is executed by a processor, the steps of the method for adding additional annotation information during video playback as described in any one of claims 1 to 7 are implemented.

Images