Automatic video editing system based on Internet hotspots

Abstract

The invention discloses an automatic video editing system based on Internet hotspots, and the system comprises a video understanding module which carries out the content understanding of one or more videos inputted by a user, and obtains the content, segment and transition content of each video; the network topic selection module is used for selecting topics according to the video content of the video understanding module in combination with network hotspots and hot words, and performing network duplicate checking on the selected topics; and the dynamic generation module is used for dynamically selecting proper materials according to the questions selected by the network question selection module and packaging the materials into sheets. According to the scheme, the video auditing passing rate of the selected questions on all large platforms is guaranteed, the most suitable materials are selected to be edited into pieces according to all the selected questions, and one-button and batch short video production is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated media video production, and in particular to an automatic video editing system based on Internet hot spots. Background Art

[0002] In the field of integrated media video production, topic selection and video editing are complex and repetitive tasks. For example, given a variety show, the editor needs to find highlights from the show and create short video scenes. In this process, the editor not only needs to fully understand the video content and extract potential highlights from the video, but also needs to combine online hot spots, hot words and check for duplicates to reasonably select topics to ensure the popularity of the video. At the same time, the editor should make full use of the program materials, try to discover topics, ensure the number of videos, and ultimately promote the topicality of the program.

[0003] General automatic video editing technology usually edits the video with specified relevant content, such as specifying characters, manuscripts, etc. to ensure the quality of the edited video. Its method still strongly relies on the editor's familiarity with the video and current hot topics. However, how to automatically integrate hot research, topic selection, duplicate checking, and film production, and truly achieve one-click, batch production of short videos, so that editors only need to select and review short videos, is still a problem that technicians in this field need to solve. Summary of the invention

[0004] In view of the above technical problems, the present invention provides an automatic video editing system based on Internet hot spots.

[0005] The present invention is implemented by adopting the following technical scheme: a video automatic editing system based on Internet hot spots, comprising the following modules: Video understanding module: understands the content of one or more videos input by the user, and understands the content, segmentation, and transition content of each video; Online topic selection module: select topics based on the video content of the video comprehension module combined with online hot spots and hot words, and check the selected topics for online duplication; Dynamic generation module: dynamically selects appropriate materials based on the topic selection module of the network, and packages them into films.

[0006] Specifically, the video understanding module understands the video content and converts the video content into text, and includes multiple submodules for processing input videos, specifically including: Transition division submodule: Use computer vision algorithm to divide the video into transitions according to the degree of picture changes; Image vector submodule: samples each divided transition frame and generates key frames into image semantic vectors using a multimodal model; Speech recognition submodule: converts video speech into text through ASR automatic speech recognition technology as the original corpus of the video; Scene recognition submodule: Through the scene recognition model, each transition in the input video is identified to which scene it belongs and whether it is a famous landmark.

[0007] Specifically, the submodule for processing the input video also includes: Subtitle recognition submodule: Use OCR technology to identify prominent text and key text information appearing in the video according to the timeline; Face recognition submodule: outputs information about the appearance of people in a timeline; Video understanding submodule: understand the video content; Text vectorization submodule: stores text vectorization for each transition, segment, and video description of the video.

[0008] Specifically, the video understanding submodule understands the video content including: Use a large video understanding model to sample each transition and describe the content of each transition; Use a large language model to segment the video according to the timeline and combine the images, characters, voices, scenes, landmarks, and subtitles of each transition; Use a large language model, combined with segment division and each transition in the segment, to describe the content of each segment; Use a large language model to combine the content of the segment and each transition under the segment to describe the content of each transition; Use a large language model to combine all the clips in the above video to describe the content of the entire video and generate classifications and labels.

[0009] Specifically, the network topic selection module includes the following submodules: Hot words extraction submodule: obtain current hot topics unrelated to the topic, extract relevant topics on social media platforms, and use search hot words related API to extract current hot words; Topic planning submodule: Use a large language model to integrate the program and segment descriptions of all materials, and combine the hot words to pull the submodule network real-time information to generate a group of short video topics to be selected; Online duplication check submodule: For the topics generated by the topic planning submodule, titles are checked for duplication on major short video platforms.

[0010] Specifically, the duplicate checking submodule of the network includes: determining whether the topic already exists on the network based on the search results of major platforms, combining semantic similarity and word segmentation retrieval methods, and forming a final topic list.

[0011] The beneficial effects of the present invention are as follows: the present invention automatically understands one or a group of videos input by the user, and forms three-level corpus of transitions, segments, and programs as well as vectorized storage, selects topics based on video content, online hot spots, etc., and ensures the video review pass rate of the selected topics on major platforms through duplicate checking, selects the most suitable materials for each topic, and edits them into films, thus realizing one-click, batch production of short videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0013] Figure 1 This is a diagram of the architecture of the automatic video editing system based on Internet hot spots of the present invention; Figure 2 This is a flow chart of the video understanding stage in an embodiment of the present invention; Figure 3 This is a flow chart of the topic selection stage in an embodiment of the present invention; Figure 4 A flow chart of the generation phases in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0015] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0016] The following is combined with Figures 1 to 4 , some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0017] The present invention proposes a video automatic editing system based on Internet hot spots, such as Figure 1 As shown, it includes the following modules: Video understanding module: understands the content of one or more videos input by the user, and understands the content, segmentation, and transition content of each video; Online topic selection module: select topics based on the video content of the video comprehension module combined with online hot spots and hot words, and check the selected topics for online duplication; Dynamic generation module: dynamically selects appropriate materials based on the topic selection module of the network, and packages them into films.

[0018] In this embodiment, the video understanding module understands the video content and converts the video content into text, including multiple submodules for processing input videos, specifically including: Transition division submodule: Use computer vision algorithm to divide the video into transitions according to the degree of picture changes; Image vector submodule: samples each divided transition frame and generates key frames into image semantic vectors using a multimodal model; Speech recognition submodule: converts video speech into text through ASR automatic speech recognition technology as the original corpus of the video; Scene recognition submodule: Through the scene recognition model, each transition in the input video is identified to which scene it belongs and whether it is a famous landmark.

[0019] The submodules for processing input video also include: Subtitle recognition submodule: Use OCR technology to identify prominent text and key text information appearing in the video according to the timeline; Face recognition submodule: outputs information about the appearance of people in a timeline; Video understanding submodule: understand the video content; Text vectorization submodule: stores text vectorization for each transition, segment, and video description of the video.

[0020] In this embodiment, the video understanding submodule understands the video content specifically including: Use a large video understanding model to sample each transition and describe the content of each transition; Use a large language model to segment the video according to the timeline and combine the images, characters, voices, scenes, landmarks, and subtitles of each transition; Use a large language model, combined with segment division and each transition in the segment, to describe the content of each segment; Use a large language model to combine the content of the segment and each transition under the segment to describe the content of each transition; Use a large language model to combine all the clips in the above video to describe the content of the entire video and generate classifications and labels.

[0021] In this embodiment, the network topic selection module includes the following submodules: Hot words extraction submodule: obtain current hot topics unrelated to the topic, extract relevant topics on social media platforms, and use search hot words related API to extract current hot words; Topic planning submodule: Use a large language model to integrate the program and segment descriptions of all materials, and combine the hot words to pull the submodule network real-time information to generate a group of short video topics to be selected; Online duplication check submodule: For the topics generated by the topic planning submodule, titles are checked for duplication on major short video platforms.

[0022] Specifically, the duplicate checking submodule of the network includes: determining whether the topic already exists on the network based on the search results of major platforms, combining semantic similarity and word segmentation retrieval methods, and forming a final topic list.

[0023] The present invention also correspondingly proposes a video automatic editing method based on Internet hot spots, comprising the following steps: Step S1: The user inputs the video to be edited, and the system video understanding module understands the video content, forms three-level corpus of transition, segment and program, and stores them in vector form; Step S2: Select topics and check for duplicate content based on video content and online hot topics to form a final topic list; Step S3: According to each topic, the most suitable material is selected for editing into a film. Based on the video, a customized template packaging of the program is added to form the final product.

[0024] In this embodiment, step S3 includes the following sub-steps: Step S31: vectorizing the text of the topic, and using text similarity to search for transitions within a preset range in the input video as creative materials for the topic; Step S32: using the big language model, input the topic and material text, and use the big model to select transitions and design key shots for the topic; Step S33: using the multimodal model, converting the key shot design text designed by the large language model into a vector, and circling the shot screen entry and exit points that best match the design in the selected transition; Step S34: Finally, the selected transitions and the final entry and exit points in each transition are spliced ​​in order to form a final video.

[0025] Specifically, the step S33 further includes: using the time point of the transition speech text automatically recognized by the speech recognition submodule ASR to perform time correction on the entry and exit points.

[0026] In one embodiment, the processing and editing of the video in the present invention is mainly divided into the following stages: Understanding stage: Figure 2 As shown, the system understands the content of one or more videos input by the user, and understands the content, segmentation, and transition content of each video.

[0027] Topic selection stage: Figure 3 As shown, the system selects topics based on the video content combined with online hot spots and hot words, and checks the selected topics for duplication on the Internet.

[0028] Generation phase: Figure 4 As shown, the system dynamically selects appropriate materials based on the topic and packages them into a film.

[0029] The goal of the understanding phase is to make the system understand the video content and convert it into text, which is convenient for the unified design of the large language model. Each input video will be processed as follows (the whole process is as follows Figure 1 shown): 1. Transition division submodule: Through computer vision algorithm, the video is divided into transitions according to the degree of picture change.

[0030] 2. Image vector submodule: Sample frames for each transition and use a multimodal model to generate image semantic vectors from key frames.

[0031] For example, for each of the above transitions, a sampled picture is formed by taking one frame every three frames, and each picture is used with the CN-CLIP model to generate a picture semantic vector for each picture (such as a vector of [-0.0192, 0.0559, 0.0147, 0.0041, 0.0461, 0.0098, 0.0115, 0.0014, 0.0174, -0.0151, ...]) and store it in the vector database.

[0032] 3. Speech recognition submodule: The video speech is transcribed into text through ASR technology to form the original corpus of the video. Furthermore, if the event or variety show records the voiceprint, the name of the speaker is added to the original corpus. This submodule finally outputs the speech text of each speaker in the video in chronological order. For example, "00:01:03-00:01:12 Zhang XX: The children have experienced several consecutive days of high-intensity training."

[0033] 4. Scene and landmark recognition submodule: Through the scene recognition model, each transition in the video is identified to which scene it belongs and whether it is a famous landmark. For example, a video may produce the following information after this step: desert, outdoor.

[0034] 5. Subtitle recognition submodule: Use OCR technology to identify conspicuous text in the video according to the timeline, with the goal of extracting key text information such as game segments of variety shows, guest names, package subtitles, etc. For example, in the frame shown in the figure, we extract: "00:07:31-00:07:33 Zhongwei is the ancient Silk Road".

[0035] 6. Face recognition submodule: If the user further uses face recognition technology, this submodule can generate information about the appearance of people in a timeline. For example, the generated information is: "00:03:04-00:03:12 Zhang XX".

[0036] 7. Video understanding submodule: This submodule contains 3 sub-processes to understand the video content.

[0037] A. Use a large video understanding model (such as Qwen2-VL) to sample each transition and describe the content of each transition.

[0038] For example, a rider driving a white quad bike on an off-road track, the video understanding model can generate the following image: Scene type: close-up; Scene description: The video shows a rider driving a white quad bike on an off-road track. Some spectators and billboards can be seen in the background, and there are trees and some spectators watching the race around the track. The rider is wearing a blue protective suit and helmet, looking very professional.

[0039] B. Use the big language model to segment the video according to the timeline and combine the picture, character, voice, scene, landmark, and subtitle information of each transition. For example, input the following numbered transition description text into the big model, and ask the big model to output the transition number of the segment division point.

[0040] enter: “ The first transition: Image: A close-up of a rider on a white quad bike on an off-road track. Some spectators and billboards can be seen in the background, and the track is surrounded by trees and some spectators watching the race. The rider is wearing a blue protective suit and helmet, looking very professional.

[0041] Voice: Confined to the world of motorcycles.

[0042] Text: Confined to the world of motorcycles.

[0043] Second transition: Screen:... ... Transition 50: Screen:... voice:... ” The large model outputs: [1, 13, 34, 50], which means that transitions 1 to 12 are one segment, transitions 13 to 33 are one segment, and transitions 34 to 50 are one segment.

[0044] C. Use a large language model, combined with the above segment division and each transition in the segment, to describe the content of each segment. For example: "Segment 1: In the program, someone is receiving motorcycle training."

[0045] D. Use the large language model to combine the content of the segment and each transition below it to describe the content of each transition. For example: "Transition 1: Someone is driving a white four-wheeled motorcycle in the sand."

[0046] E. Use a large language model to combine all the clips in the above video to describe the content of the entire video and generate categories and tags. For example: "Video description: This video is a variety show, which includes sand motorcycle training, ..., and other links. Category: Variety show. Tags: outdoor, desert, motorcycle, children's variety show".

[0047] 8. Text vectorization module: Text vectorization storage for each transition, segment and video description of the video.

[0048] At this point, the system video understanding module has automatically understood one or a group of videos input by the user, and formed three-level corpus of transitions, clips, and programs as well as vectorized storage.

[0049] The goal of the topic selection stage is to select topics based on the video content, combined with online hot spots and hot words, and then conduct online duplication check on this basis. This part has the following processing (the whole process is as follows Figure 2 shown): 1. Hot spots and hot words extraction: This part integrates the following information: (1) Objective hotspot extraction: This part obtains current hotspots that are not related to the topic, and obtains current hotspots through data sources such as Baidu real-time hotspots.

[0050] (2) Extracting related hot topics: This part extracts relevant topics on social media platforms based on the classification and labels of the programs when understanding the video.

[0051] (3) Hot word extraction: This part uses the search hot word related API to extract the current hot words.

[0052] According to the above process, the following text is formed: “ Current hot spots: Opening the Big Snow Festival with Poetry ... Current hot topics related to this video: Even a 6-year-old knows not to be a mama's boy ... ” 2. Topic planning: This part uses a large language model to integrate the program and segment descriptions of all materials, and combines the above-mentioned real-time network information to generate a set of short video topics to be selected. For example, the content input into the large language model is as follows: " Video 1 content: This video is a variety show, which includes sand motorcycle training, ..., and other segments.

[0053] Video 1 Clip 1: In a certain program, someone is receiving motorcycle training Video 1 Clip 2: ... Current hot spots: Opening the Big Snow Festival with Poetry ... Current hot topics related to this video: Even a 6-year-old knows not to be a mama's boy ... " The topics output by the large model are shown in the following example: ['Desert Adventure Tour', 'Ancient Trade Desert Bazaar', ...].

[0054] 3. Online duplication check: This part checks the titles of the above-mentioned topics on major short video platforms. The duplication check method is based on the search results of major platforms, combined with semantic similarity and word segmentation search methods to determine whether the topic already exists on the Internet, and form a final list of topics.

[0055] So far, the system's online topic selection module has selected topics based on video content, online hot spots, etc., and has ensured the video review pass rate on major platforms through duplication checking.

[0056] The purpose of the generation phase is to select the most appropriate material for each topic and edit it into a film. This part has the following processing (the whole process is as follows Figure 3 shown): 1. Perform text vectorization on the above topics, and use text similarity to find the top 50 transitions in the input video as creative materials for the topic. For example, for the topic of "Desert Adventure Tour", by calculating the text similarity with the transitions, the following list of transition materials suitable for the topic is finally formed: "Transition 1: Someone is driving a white quad bike in the sand, Transition 2: ..., Transition 50: All the contestants celebrate at the finish line." 2. Use the big language model to input the topic and material text, and use the big model to select transitions and design key shots based on the topic. The input big model content is shown in the following example: “ Theme: Desert Adventure Tour material: Transition 1: Someone is driving a white four-wheeled motorcycle in the sand. Transition 2: ... Transition 50: All runners celebrate at the finish line.

[0057] ” For example, the output of a large model is: [{"ref": 1, "text": "The moment of starting a motorcycle (close-up)"}, {"ref":5, "text": "The motorcycle performs a drift in the sand (close-up)"}, ... , {"ref": 50, "text": "The contestants high-five each other (panoramic)"}]. "ref" represents the transition material number used in each designed shot, and "text" represents the key content that the model hopes to select from the material.

[0058] 3. Using the multimodal model, the key shot design text designed by the large language model is converted into a vector, and the entry and exit points of the shot that best matches the design are circled in the selected transition. On this basis, the ASR text timing of this transition is referenced to correct the entry and exit points in time to prevent speech truncation.

[0059] For example, for the design lens {"ref": 1, "text": "The moment of starting the motorcycle (close-up)"} in the above example, the solution would be: First, the CN-CLIP model is used to convert the "moment of starting the motorcycle (close-up)" into a vector, and the similarity is calculated with all the sampled frames of transition 1 in the video (step #2 of the understanding stage), and the time point with the greatest similarity is selected. Assuming that the time point with the greatest similarity in this example is "00:01:35.500", the shots before and after 1.5 seconds will be used here, that is, "00:01:34.0 ~ 00:01:37.0".

[0060] At the same time, refer to the voice text within this period and make appropriate corrections. For example, if the time point of a voice speech is "00:01:36.0~00:01:38.0", the segment here will eventually be corrected to "00:01:34.0~00:01:38.0".

[0061] 4. Finally, the selected transitions and the final entry and exit points of each transition are spliced ​​in order to form the final video.

[0062] 5. Based on the video, add customized template packaging for the program to form the final product.

[0063] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.

[0064] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. An automatic video editing system based on Internet hot spots, characterized in that: Includes the following modules: Video understanding module: understands the content of one or more videos input by the user, and understands the content, segmentation, and transition content of each video; Online topic selection module: select topics based on the video content of the video comprehension module combined with online hot spots and hot words, and check the selected topics for online duplication; Dynamic generation module: dynamically selects appropriate materials based on the topic selection module of the network, and packages them into films.

2. The automatic video editing system based on Internet hot spots as claimed in claim 1, characterized in that: The video understanding module understands the video content and converts the video content into text, and includes multiple submodules for processing input videos, specifically including: Transition division submodule: Use computer vision algorithm to divide the video into transitions according to the degree of picture changes; Image vector submodule: samples each divided transition frame and generates key frames into image semantic vectors using a multimodal model; Speech recognition submodule: converts video speech into text through ASR automatic speech recognition technology as the original corpus of the video; Scene recognition submodule: Through the scene recognition model, each transition in the input video is identified to which scene it belongs and whether it is a famous landmark.

3. The automatic video editing system based on Internet hot spots as claimed in claim 2, characterized in that: The submodule for processing input video also includes: Subtitle recognition submodule: Use OCR technology to identify prominent text and key text information appearing in the video according to the timeline; Face recognition submodule: outputs information about the appearance of people in a timeline; Video understanding submodule: understand the video content; Text vectorization submodule: stores text vectorization for each transition, segment, and video description of the video.

4. The automatic video editing system based on Internet hot spots as claimed in claim 3, characterized in that: The video understanding submodule understands the video content specifically including: Use a large video understanding model to sample each transition and describe the content of each transition; Use a large language model to segment the video according to the timeline and combine the images, characters, voices, scenes, landmarks, and subtitles of each transition; Use a large language model, combined with segment division and each transition in the segment, to describe the content of each segment; Use a large language model to combine the content of the segment and each transition under the segment to describe the content of each transition; Use a large language model to combine all the clips in the above video to describe the content of the entire video and generate classifications and labels.

5. The automatic video editing system based on Internet hot spots as claimed in claim 1, characterized in that: The network topic selection module includes the following submodules: Hot words extraction submodule: obtain current hot topics unrelated to the topic, extract relevant topics on social media platforms, and use search hot words related API to extract current hot words; Topic planning submodule: Use a large language model to integrate the program and segment descriptions of all materials, and combine the hot words to pull the submodule network real-time information to generate a group of short video topics to be selected; Online duplication check submodule: For the topics generated by the topic planning submodule, titles are checked for duplication on major short video platforms.

6. The automatic video editing system based on Internet hot spots as claimed in claim 5, characterized in that: The duplicate checking submodule of the network includes: determining whether the topic already exists on the network based on the search results of various platforms, combining semantic similarity and word segmentation retrieval methods, and forming a final topic list.

Images