A video fast editing method based on artificial intelligence technology

By dividing the video material into independent segments and generating description text, combining user prompt information and logical judgment model, the problem of insufficient logic of video stitching in the existing technology is solved, and video clips with strong logic and coherent content are generated Effect.

CN117692676BActive Publication Date: 2024-11-15GUANGDONG INSIGHT BRAND MARKETING GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311688242.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-08
Publication Date
2024-11-15
Estimated Expiration
2043-12-08

AI Technical Summary

Technical Problem

The existing artificial intelligence video editing technology produces poor editing videos in the splicing method, and the plot of the video content is not coherent enough, which seems abrupt.

Method used

By dividing the video material into independent video clips and storing it vectorized, the video description text is generated using the open source big model of video to text, and vectorized search is performed based on the prompt information input by the user to filter out video scripts with certain logic. , and use the logical judgment model to optimize the video script, and finally stitch the video clips according to the description text sequence to generate the final edited video.

Benefits of technology

The generated videos are highly logical and coherent, which solves the problem of insufficient logic of splicing videos in the prior art and improves the overall effect of the video.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117692676B_ABST
    Figure CN117692676B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for rapid video editing based on artificial intelligence technology, including steps such as video material processing, video retrieval and video splicing. The present invention slices and segments any video material into relatively independent and complete video clips, and simultaneously provides corresponding description texts, and stores them in a database in the form of vectors, so that the aforementioned video clips can be retrieved and spliced ​​in the form of video scripts, thereby generating a spliced ​​video with strong logic and coherent content.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to video processing technology, and more specifically, to a video editing method. Background Art

[0002] The dissemination and consumption of video content on major Internet platforms continues to grow. Video editing, as an important video processing method, can cut, edit and splice original video materials to create a more attractive and effective visual presentation.

[0003] Based on the development of Internet technology and artificial intelligence, various video editing methods and tools that use artificial intelligence technology for video processing have emerged. Through these current video processing methods and tools using artificial intelligence, it is possible to generate, edit, segment, splice videos, and automatically synthesize voice by inputting text content. Currently, artificial intelligence video processing software on the market includes Douyin AI Painting, Zhiying, VidPress, VideoLab, Wanna DoVideo, etc.

[0004] Compared with manual video editing, users can create video content without particularly tedious manual operations, which improves efficiency to a certain extent.

[0005] However, the existing artificial intelligence video editing technology has relatively weak logic for edited videos generated by splicing, and the plot of the video content is not coherent and is relatively abrupt. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for rapid video editing based on artificial intelligence technology to solve the problem of poor logic of the edited videos generated by splicing in the prior art.

[0007] In order to achieve the above-mentioned purpose, the technical scheme of the present invention is: a method for fast video editing based on artificial intelligence technology, comprising: (1) using a video segmentation method to slice any video material into several relatively independent and complete video segments, thereby obtaining a video material library; vectorizing each video segment in the video material library to form a video vector and storing it in a video vector database; using an open source large model of video to text to describe the content of each video segment to obtain a corresponding video description text, vectorizing the video description text to form a text vector and storing it in the video vector database; (2) vectorizing the editing prompt information input by the user, and obtaining a text vector from the video vector database; The first N video description texts most relevant to the editing prompt information are found in the database; the first N video description texts are used as contexts, and M description texts with certain logic are screened out using a large language model to form a video script, and then a logic judgment model is used to judge the logic of the video script to obtain a video script with certain logic; according to each video description text in the video script, the corresponding video clip is retrieved from the video vector database to obtain M video clips; (3) the M video clips are spliced ​​according to the sequence of the video description texts to obtain a final editing video; preferably, transition clips can be added between the spliced ​​video clips.

[0008] Preferably, the step of using the logic judgment model to judge the logic of the video script includes: setting a threshold, and when the logic probability value is less than the threshold, regenerating the video script until the logic probability value is greater than the threshold; or stopping when the number of attempts exceeds a certain number, and taking the video script with the largest logic probability value as the final result.

[0009] Preferably, the training method of the logic judgment model is as follows: randomly extracting n paragraphs of text from a story article, and using a large language model to summarize each paragraph of text into a sentence, and extracting n video clips from manually edited video clips, and using video-chatgpt to describe each video clip as a sentence; arranging the n sentences in order to construct positive samples, and randomly shuffling the n sentences to construct negative samples, with a positive to negative sample ratio of 1 to 5; using the text encoder in clip4clip to convert each sentence into a vector with consistent dimension, and using a transformer-based sequence classification model to classify the text sequence to determine whether it is logical; the loss function adopts cross entropy loss.

[0010] Preferably, the clip prompt information includes a theme word and a theme element, and each retrieval uses the theme word plus the theme element to find the first N relevant video descriptions.

[0011] Compared with the prior art, the present invention slices and segments any video material into relatively independent and complete video clips, and provides corresponding description texts, and stores them in a database in the form of vectors, so that the aforementioned video clips can be retrieved and spliced ​​through video scripts, thereby generating a spliced ​​video with strong logic and coherent content. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The structure of the present invention and its beneficial technical effects are described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0013] Figure 1 It is a schematic diagram of the process of the present invention;

[0014] Figure 2 It is a schematic diagram of the video material processing flow of the present invention;

[0015] Figure 3 It is a schematic diagram of the video retrieval process of the present invention. DETAILED DESCRIPTION

[0016] In order to make the invention purpose, technical solution and beneficial technical effects of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described in this specification are only for explaining the present invention, not for limiting the present invention.

[0017] See also Figures 1 to 3 The video fast editing method based on artificial intelligence technology in this embodiment includes three parts: video material processing, video retrieval and video splicing.

[0018] (I) Video material processing:

[0019] Use video segmentation methods such as PySceneDetect to slice the video material and divide it into N relatively independent and complete video segments, thereby obtaining a video material library; use the existing video multimodal pre-trained large-scale model to vectorize the video segments, form video vectors and store them in the video vector database for subsequent rapid retrieval.

[0020] Use video-chatgpt to describe the content of each video clip in detail and obtain the video description text of each video. Then use the pre-trained video text multimodal pre-trained large model to vectorize the video description text, and store the video description text vector in the video vector database to facilitate the subsequent rapid retrieval of the specified video based on the text content.

[0021] The specific video vectorization method uses the text encoder in the video-text contrastive learning encoder clip4clip, and the vector database uses faiss.

[0022] (II) Video retrieval:

[0023] The user inputs the prompt information that he wants to edit, including the theme words and theme elements, such as "joy", "horror", etc., and the theme elements to be included, such as family, zombies, etc. The theme words and related theme elements are vectorized using the text encoder in clip4clip, and the top N most relevant video descriptions are found from the video text description. Here, each search uses the main word plus a theme element to find the top N relevant video description texts. In the end, if there are 10 elements and 1 theme word, 10N relevant video descriptions are obtained.

[0024] Take 10N related video descriptions as context, let a large language model such as chatgpt select M descriptions from them, and sort them into a video description text sequence with certain story and logic containing keywords and elements to obtain a video script. After each description in the video script is vectorized, the corresponding video is retrieved from the video database. Since the video must be in the video library, the video with the greatest similarity is the video corresponding to the text description, and finally M videos are obtained.

[0025] Since the logic of the description sequence screened and sorted by the large language model may not be strong, the present invention trains a logic judgment model to judge whether the description sequence has sufficient logic. Specifically, it includes: randomly extracting n paragraphs of text from the story article, and using the large language model to summarize each paragraph of text into a sentence, and extracting n video clips from the manually edited video clips, and using video-chatgpt to describe the video clips into a sentence; and arranging the n sentences in order to construct positive samples, and randomly shuffling the n sentences to construct negative samples, with a positive to negative sample ratio of 1 to 5; each sentence will be converted into a vector with consistent dimensions using the text encoder in clip4clip, and the transformer-based sequence classification model will be used to classify the text sequence to determine whether it is logical; the loss function uses cross entropy loss. In specific use, after the large language model filters and sorts the descriptions, the logical judgment model is used to judge the results and a threshold is set. When the logical probability value is less than the threshold, it is regenerated until the generated logic exceeds the set threshold; or when the number of attempts exceeds a certain number, it stops and takes the description with the largest logical probability value as the final result.

[0026] (III) Video stitching:

[0027] The retrieved video clips are spliced ​​according to the video description text sequence, and finally a video clip that conforms to the video script is obtained. In order to make the splicing more natural, the open source video transition tool gl-transition is used for smooth processing to make the video transition more natural. In order to match the most suitable music for the video, video2music can be used to generate suitable background music for the video.

[0028] According to the disclosure and teaching of the above description, those skilled in the art to which the present invention belongs may also make appropriate changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although some specific terms are used in this specification, these terms are only for the convenience of description and do not constitute any limitation to the present invention.

Claims

1. A video fast editing method based on artificial intelligence technology, characterized in that: include: (1) Slice any video material using a video segmentation method to divide it into several relatively independent and complete video segments, thereby obtaining a video material library; vectorize each video segment in the video material library to form a video vector and store it in a video vector database; use an open source large model of video to text to describe the content of each video segment to obtain a corresponding video description text, vectorize the video description text, form a text vector and store it in the video vector database; (2) after vectorizing the editing prompt information input by the user, finding the top N video description texts most relevant to the editing prompt information from the video vector database; The first N video description texts are used as contexts, and M description texts with certain logic are screened out by using a large language model to form a video script, and then a logic judgment model is used to judge the logic of the video script to obtain a video script with certain logic; According to each video description text in the video script, a corresponding video clip is retrieved from the video vector database to obtain M video clips; (3) splicing the M video clips according to the sequence of the video description text to obtain a final edited video.

2. The method for rapid video editing based on artificial intelligence technology according to claim 1, characterized in that: The step of using the logic judgment model to judge the logic of the video script includes: setting a threshold, and when the logic probability value is less than the threshold, regenerating the video script until the logic probability value is greater than the threshold; or stopping when the number of attempts exceeds a certain number, and taking the video script with the largest logic probability value as the final result.

3. The method for rapid video editing based on artificial intelligence technology according to claim 2 is characterized in that: The training method of the logic judgment model is as follows: randomly extract n paragraphs of text from a story article, and use a large language model to summarize each paragraph of text into a sentence, and extract n video clips from manually edited video clips, and use video-chatgpt to describe each video clip as a sentence; arrange the n sentences in order to construct positive samples, and randomly shuffle the n sentences to construct negative samples, with a positive to negative sample ratio of 1 to 5; use the text encoder in clip4clip to convert each sentence into a vector with consistent dimension, and use a transformer-based sequence classification model to classify the text sequence to determine whether it is logical; the loss function uses cross entropy loss.

4. The method for rapid video editing based on artificial intelligence technology according to claim 1, characterized in that: The clip prompt information includes a theme word and a theme element. Each time a search is performed, the theme word and the theme element are used to find the first N related video descriptions.