Method, medium and device for automatically generating short video manuscripts with complex event context

Abstract

The present invention provides a method, medium, and device for automatically generating short video manuscripts based on complex event sequences. The method comprises the following steps: S1, segmenting the original video by scene and converting it into video vectors to form a video vector library; S2, converting the complex event sequence into an event sequence; S3, performing a video search within the video vector library based on the event sequence and synthesizing the search results into a short video manuscript. The present invention can automatically generate videos based on the development of an event, facilitating a quick understanding of the event's development. Furthermore, the present invention can automatically collect video footage based on the event sequence, saving time in video production.

Description

Technical Field

[0001] The present invention relates to the field of text processing and analysis technology, and in particular to a method, medium, and device for automatically generating short video manuscripts with complex event context. Background Art

[0002] Major social events frequently occur in today's society, significantly impacting policy, the economy, culture, technology, and other areas. These events continuously evolve and ferment, forming a complex and intricate web of events. Understanding the development of complex events can provide insight into social developments and improve our ability to respond to and handle major events and emergencies.

[0003] When understanding the dynamics of complex events, the traditional approach is to search for and read relevant news materials. On the one hand, news materials are not integrated, making collection time-consuming. On the other hand, news materials are often presented in text form, which can be tedious to read. Presenting the development of complex events in the form of short videos is clear and intuitive, and more in line with the current usage habits of social media users. Therefore, a method for automatically generating short video transcripts of complex event dynamics is needed. Summary of the Invention

[0004] The present invention aims to provide a method, medium and device for automatically generating short video manuscripts with complex event context to solve the above-mentioned problems.

[0005] The present invention provides a method for automatically generating short video manuscripts with complex event context, comprising:

[0006] Constructing a video vector library: Segment the original video by scene and convert it into video vectors to form a video vector library;

[0007] Complex event context serialization: converting complex event context into event sequence;

[0008] Video recall and synthesis: Perform video retrieval in the video vector library according to the event sequence, and synthesize the retrieval results into short video manuscripts.

[0009] (1) Construction of video vector library

[0010] Building a video vector library involves segmenting the original video by scene and converting the video segments into video vectors using the multimodal universal video model InterVideo. Because events themselves contain limited information, they may correspond to a small video segment. Segmenting videos by scene allows long videos to be rationally divided into short segments, facilitating the establishment of a correspondence between events and videos.

[0011] (2) Complex event sequence sequence

[0012] Complex event sequences are intertwined and complex in structure, typically represented as directed graphs, while short videos themselves are sequences of image frames. To construct short videos depicting complex event sequences, it is necessary to convert the complex event sequences represented by directed graphs into event sequences. This method uses a breadth-first traversal of the graph to convert complex event sequences into event sequences.

[0013] (3) Video Recall and Synthesis

[0014] Video recall involves using the multimodal general video model InterVideo to convert the event text into a semantically aligned text vector for each event in the event sequence. Based on the text vector, the relevant video vector is then retrieved from the video vector library. Video synthesis involves finding the corresponding video clip based on the recalled video vector and synthesizing the video clips into a complete video transcript according to the order of the event sequence.

[0015] The present invention also provides a computer terminal storage medium storing computer terminal executable instructions, wherein the computer terminal executable instructions are used to execute the above-mentioned method for automatically generating short video manuscripts with complex event context.

[0016] The present invention further provides a computing device, comprising:

[0017] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned method for automatically generating short video manuscripts with complex event context.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0019] 1. The present invention can automatically generate videos according to the context of event development, providing convenience for quickly understanding the development of events.

[0020] 2. The present invention can automatically collect video materials according to the context of events, saving time in video production. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1Flowchart of the method for automatically generating short video manuscripts with complex event context in an embodiment of the present invention

[0023] Figure 2 This is a flowchart of constructing a video vector library in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0026] Example

[0027] like Figure 1 As shown, this embodiment proposes a method for automatically generating short video manuscripts with complex event context, including the following steps:

[0028] S1, divides the original video into scenes and converts it into video vectors to form a video vector library; Figure 2 As shown, specifically:

[0029] S11, for original video , use the scene segmentation tool to segment the original video by scene Split and obtain a collection of video clips ;

[0030] S12, for a video segment collection Video clips from , using the multimodal general video model InterVideo to convert it into a video vector ;

[0031] S13, the corresponding video clip collection All video vectors are stored in the video vector library, and the corresponding relationship between the video vector and the video clip is established.

[0032] S2, transforming complex event context into event sequence; specifically:

[0033] S21, constructing a directed graph based on complex event context;

[0034] S22, using entity recognition algorithm to extract the occurrence time of each event, and select the earliest event node as the starting point of directed graph traversal;

[0035] S23, initialize all nodes to be unvisited and initialize an empty queue;

[0036] S24, starting from the starting point, visiting the starting point and marking it as visited, and adding the starting point to the queue;

[0037] S25, if the queue is not empty, continue execution;

[0038] S26, dequeue the first node in the queue, visit all unvisited adjacent nodes in sequence, mark them as visited and add them to the queue, and return to step S25;

[0039] S27, when the queue in step S25 is empty, the entity recognition algorithm ends and an event sequence is obtained. .

[0040] S3, according to the event sequence, performs video retrieval in the video vector library and synthesizes the retrieval results into short video manuscripts; specifically:

[0041] S31, for the event sequence Every event in , using the multimodal general video model InterVideo to convert it into a text vector ;

[0042] S32, the text vector Calculate the cosine similarity with each video vector in the video vector library. If the cosine similarity is greater than the set threshold, the video clip corresponding to the event will be recalled.

[0043] S33, for each event, all corresponding video clips are synthesized into a video and the audio is removed; then the event text is converted into audio through speech synthesis as the new audio of the video;

[0044] S34, merge the videos of all events according to the order in the event sequence to form a short video manuscript.

[0045] In addition, in some embodiments, a computer terminal storage medium is proposed, which stores computer terminal executable instructions, and the computer terminal executable instructions are used to execute the method for automatically generating short video manuscripts of complex event context as described in the above embodiments. Examples of computer storage media include magnetic storage media (e.g., floppy disks, hard disks, etc.), optical recording media (e.g., CD-ROMs, DVDs, etc.) or memories, such as memory cards, ROMs, or RAMs. Computer storage media can also be distributed on network-connected computer systems, such as application stores.

[0046] Furthermore, in some embodiments, a computing device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for automatically generating a short video manuscript of a complex event context as described in the above embodiments. Examples of computing devices include a PC, a tablet computer, a smartphone, or a PDA.

[0047] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for automatically generating short video manuscripts with complex event context, characterized in that: The steps include: S1, divides the original video into scenes and converts them into video vectors to form a video vector library; S2, transforming complex event context into event sequence; S3, according to the event sequence, performs video retrieval in the video vector library and synthesizes the retrieval results into short video manuscripts; Step S1 includes the following sub-steps: S11, for original video , use the scene segmentation tool to segment the original video by scene Split and obtain a collection of video clips ; S12, for a video segment collection Video clips from , using the multimodal general video model InterVideo to convert it into a video vector ; S13, the corresponding video clip collection All video vectors are stored in the video vector library, and the corresponding relationship between the video vector and the video clip is established; Step S2 includes the following sub-steps: S21, constructing a directed graph based on complex event context; S22, using entity recognition algorithm to extract the occurrence time of each event, and select the earliest event node as the starting point of directed graph traversal; S23, initialize all nodes to be unvisited and initialize an empty queue; S24, starting from the starting point, visiting the starting point and marking it as visited, and adding the starting point to the queue; S25, if the queue is not empty, continue execution; S26, dequeue the first node in the queue, visit all unvisited adjacent nodes in sequence, mark them as visited and add them to the queue, and return to step S25; S27, when the queue in step S25 is empty, the entity recognition algorithm ends and an event sequence is obtained. ; Step S3 includes the following sub-steps: S31, for the event sequence Every event in , using the multimodal general video model InterVideo to convert it into a text vector ; S32, the text vector Calculate the similarity with each video vector in the video vector library. If the similarity is greater than the set threshold, the video clip corresponding to the event will be recalled; S33, for each event, all corresponding video clips are synthesized into a video and the audio is removed; then the event text is converted into audio through speech synthesis as the new audio of the video; S34, merge the videos of all events according to the order in the event sequence to form a short video manuscript.

2. The method for automatically generating short video manuscripts of complex event context according to claim 1 is characterized in that: The similarity is cosine similarity.

3. A computer terminal storage medium storing computer terminal executable instructions, characterized in that: The computer terminal executable instructions are used to execute the method for automatically generating short video manuscripts of complex event context as described in any one of claims 1-2.

4. A computing device, characterized in that include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for automatically generating short video manuscripts of complex event context as described in any one of claims 1-2.

Images