Intelligent video editing method based on large language model

Abstract

The invention discloses an intelligent video editing method based on a large language model, which is based on mechanism resource library materials and combines AI artificial intelligence technologies such as the large language model, a cross-modal engine, voice recognition and synthesis, an audio-visual language model and the like with video and audio production. Character manuscripts in professional media content production are processed through a large language model, video and audio materials are comprehensively and intelligently analyzed through a cross-modal engine, a voice recognition engine and the like, and intelligent lens matching is performed in a cross-modal index database and a synchronous sound index database in a semantic matching mode. And an editor can manually and quickly adjust and modify the intelligent editing result. The method can be used for intelligent production of programs such as new media short videos, event broadcast video news, television program secondary creation, movie and television play clips, collection and the like, a brand new video production mode is provided for media institutions and professional content producers, and the production requirements for mass video content under the video push system in the Internet era are met.

Description

Technical Field

[0001] The present invention relates to the technical field of video editing, and in particular to an intelligent video editing method based on a large language model. Background Art

[0002] With the continuous development of computer vision, speech recognition and other technologies, the application of cross-modal large models is becoming more mature. Such models can process data of different modalities (such as text, images, speech, etc.), realize the fusion and interaction of multimodal information, and provide more possibilities for artificial intelligence applications. From the scale of models, the diversification of application scenarios, advanced technological innovation, to the in-depth development of cross-modal large models, these trends and achievements not only reflect the great progress made in the field of artificial intelligence, but also indicate that in the future, large model technology will demonstrate its unique value and capabilities in more fields.

[0003] In the process of producing and creating audio-visual content, the information contained in the image is often not limited to visual information, but may also involve data in other modes, such as text, voice, etc. Therefore, the intelligent production of audio-visual content through cross-modal data fusion technology has become an important research and improvement direction of the present invention. With the in-depth development of media convergence, various media organizations are facing changes in media content dissemination channels and the resulting massive demand for audio-visual content. Traditional media organizations broadcast at fixed times through radio and television and other release forms, while the new Internet media releases content anytime and anywhere, and is no longer restricted by traditional broadcast channels and broadcast times. The audience's viewing channels and viewing methods have also changed significantly. Obtaining news information through the Internet has become the most common way. At the same time, because most viewers watch through fragmented time, the demand for short videos has also increased significantly.

[0004] Against the backdrop of the rapid development of new Internet media, new requirements have been put forward for the output and production efficiency of audio-visual content. However, the traditional audio-visual content production method can no longer meet the needs of the current media convergence development. Professional media organizations urgently need a new production process that is fast, efficient and can ensure content quality to meet the content production needs in the converged media environment. Summary of the invention

[0005] The present invention proposes an intelligent video editing method based on a large language model. The method is based on AI technologies such as a large language model and cross-modal analysis to automatically generate professional media content from text manuscripts, and combines the characteristics of audio-visual language to perform a series of processing such as duplicate judgment and length optimization on intelligently matched shots, thereby meeting the requirements of professional media for video production in terms of efficiency and quality.

[0006] The present invention is implemented as follows: an intelligent video editing method based on a large language model, the method steps comprising: Step 1: Perform cross-modal analysis on the material: When the material is stored, AI engines such as cross-modal models and intelligent voice will automatically perform multi-dimensional comprehensive intelligent analysis on the material; Step 2: Select the material or material group required for creation from the institution's resource library; Step 3, import the transcript and rewrite and classify it: import the video transcript and use the large language model to rewrite the video transcript into the required transcript, and classify and annotate the synchronous sound and text type. Step 4: Automatically use different intelligent matching models to match shots according to different categories of the manuscripts; Step 5, adjust the intelligent matching results of the shots and generate a matching candidate shot group: save the matching results of each sentence / paragraph as a shot group, sort them by similarity, define the maximum number of shots in the shot group, and provide the shot with the highest matching similarity in each shot group as the preferred result for the next step; Step 6, generating a sequence and adjusting it according to the audio-visual language model, including intelligently merging scenes according to the time sequence of the previous and next shots in the original material in the shot matching results and using the audio-visual language model to analyze and process the matching shot results; Step 7, generate the accompaniment and subtitles, and add music; Step 8: Complete intelligent editing and conduct manual proofreading to meet the review requirements for final release.

[0007] Furthermore, the comprehensive intelligent analysis in step 1 includes: 1.1, detect transition frames in the video, and split the continuous video material into multiple scene segments according to the detection results; 1.2, extract key frames for each video scene; 1.3, key frame cross-modal detection and analysis; 1.4, cross-modal analysis of key frames, generating vectors and saving them to the index library; 1.5, perform audio synchronous sound analysis, generate synchronous sound index and save it to the index library.

[0008] Further, in step 4, for the text content marked as simultaneous interpretation, the system performs similarity matching on the simultaneous interpretation index based on the semantic understanding of the text manuscript, including the situation where the text manuscript is not exactly the same as the text result of speech recognition, to ensure intelligent matching between the written text in the manuscript and the spoken language in the material interview and dialogue; for the main text type, the system performs matching between the text and the video-audio content in terms of vector dimension based on the semantic understanding of the text, forms similarity data according to the matching comparison result, and performs intelligent matching of shots based on the similarity.

[0009] Further, the method for intelligent scene merging according to the time sequence of the front and rear shots in the original material in step 6 is as follows: Each shot matching result contains information such as its original material id ClipID, in point IN, out point OUT, etc. Assume that the matching shot results of consecutive sentences / paragraphs of simultaneous interpretation text are C0, C1, C2... respectively; their corresponding original material ids are ClipID1, ClipID2, ClipID3... respectively, the corresponding original material in points are IN1, IN2, IN3... respectively, and the corresponding original material out points are OUT1, OUT2, OUT3... respectively; First, compare the material information of the two shots C1 and C0 in the first group, and check whether the corresponding original material IDs of the two shots are the same; If ClipID2 is different from ClipID1, the two shot matching results are from different materials, and no scene merging is required. Proceed to compare the next group of materials C2 and C1; If ClipID2 is the same as ClipID1, the continuity of the two shots also needs to be compared. Compare the in point IN2 of the material of shot C1 with the out point OUT1 of the material of shot C0; If IN2 - OUT1 < t, where t is a predefined value of the system, it indicates that the second matching shot result is continuous with the first matching shot result in time, and scene merging is performed; If IN2 - OUT1 ≥ t, where t is a predefined value of the system, it indicates that the second matching shot result is not continuous with the first matching shot result in time, and no scene merging is performed; And so on until the last simultaneous interpretation matching shot result.

[0010] Furthermore, the method of using the audio-visual language model to analyze and process the matching shot results in step 6 includes: judging the duplicate shot results of each sentence / paragraph of text; processing the shot length of the intelligently matched shot results; analyzing the audio-visual speech such as the scene size and shooting method of the front and rear shots, and matching them with the montage sentence pattern in the audio-visual language model; fine-tuning the length of the video shot according to the audio length of the dubbing or synchronous sound; after the shot matching of each sentence / paragraph of text in the manuscript, arranging and splicing the shot matching results in the order of the manuscript or the storyboard script.

[0011] Furthermore, in step 7, the voice-over part of the editing result can be generated through the speech recognition and speech synthesis engine, and subtitles can be automatically generated for the main text and the speech in the same period. The music provided by the system can be emotionally classified, and the corresponding emotion can be selected to automatically add music to the generated intelligent editing result as needed.

[0012] Furthermore, the maximum number of lenses is less than or equal to 10.

[0013] Furthermore, the step 1.4 of cross-modal analysis of key frames is: extracting the first frame of every 10 frames of the video content as a key frame, performing vector analysis on the key frame after the frame extraction, and performing difference calculation on the vectors of the two key frames before and after; If the vector difference ∂ between every two consecutive frames is less than the preset value ∆, it is considered that the scene does not need to be further split, and the vector of each key frame analyzed is stored in the index library for preservation; If the difference between two consecutive key frames is greater than or equal to the preset value ∆, the sixth frame in the segment, which is the middle frame between the two key frames, is added as a key frame, and its analysis result vector is indexed.

[0014] The beneficial effects of the present invention are: (1) Designed for the characteristics of professional media organizations, which have a large number of their own audio-visual material resources and have established their own institutional resource libraries. Intelligent editing based on institutional resource library materials can ensure the authenticity, reliability and legality of the generated content, while avoiding copyright disputes that may result from citing Internet materials.

[0015] (2) The industry's first intelligent editing mode is adopted to classify professional media manuscripts into different categories such as main text and concurrent content according to the characteristics of traditional video content production, and different AI models are used for intelligent matching according to different categories to improve the accuracy of intelligent editing.

[0016] (3) For the synchronous sound matching, semantic matching is performed through a large language model to ensure the matching degree between written and spoken language. The present invention has designed a special semantic matching mode for synchronous sound. Different from the traditional "text-speech" matching mode, the text content is first semantically understood by a large language model, and then the vector of semantic understanding is matched with the audio vector of the synchronous sound, ensuring the matching tolerance of text to speech; for professional media organizations, their manuscripts often use more formal written language, but in interviews and daily conversations, oral expressions are inevitable. Through the semantic matching mode, not only the matching problem of written text to spoken speech is solved, but also the coherence of the previous and next sentences can be intelligently judged, ensuring that the jump of the picture is avoided as much as possible in the result of text matching.

[0017] (4) Through the intelligent merging algorithm of the synchronous sound matching results, the intelligent scene merging of the synchronous sound intelligent matching lens results can be carried out, which can effectively avoid the problems of shot jump and discontinuity caused by sound matching according to separate text.

[0018] (5) Based on the large language model and cross-modal engine, intelligent shot matching of audio and video is performed. At the same time, the audio-visual language model is integrated to intelligently process the connection between shots. This not only avoids the use of repeated shots in the same program, but also can be based on the audience's viewing habits. Comprehensive information such as shot length, scene, etc. is integrated for secondary processing to form the final intelligent editing result.

[0019] (6) While AI is intelligently generating the editing results, it also automatically generates a series of candidate matching shots that have a high degree of match with the text content of the sentence / paragraph, allowing editors to quickly manually adjust and modify the intelligent editing results.

[0020] (7) The core models of the large language model and cross-modal model used in this intelligent editing system support local private deployment, which can ensure that the original material content will not be leaked during the intelligent production and creation process of the video, thereby ensuring data security.

[0021] The present invention can be widely used in the intelligent production of new media short videos, event broadcast video news, secondary creation of TV programs, trailers and highlights of film and television dramas, and other types of programs; through the application of AI intelligent technology, it provides a new video production method for various media organizations and professional content producers to meet the production requirements of massive video content under the video push system in the Internet era. It saves editors' time to browse materials, select required shots from materials, and extract lyrics from interview materials and shoot lyrics for finished films; with the help of AI dubbing, it can also save the link of dubbing by professional dubbing actors, greatly improving the production efficiency of event reporting content. For the secondary creation of finished programs, the present invention can perform intelligent analysis on finished programs, select points of interest suitable for Internet platforms in finished programs for extraction and transcription, and generate new short video manuscripts or scripts. And intelligently generate short video versions based on new points of interest to meet new needs for creation and push to different audience groups. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 It is a flow chart of the steps of the method for performing cross-modal analysis on images according to the present invention; Figure 3 It is a schematic diagram of video frame grouping according to the present invention. DETAILED DESCRIPTION

[0023] Terminology explanation: Cross-modality: Cross-modality is a process that involves extracting information from different data modalities and interactively fusing them. This includes extracting features from multiple forms of data such as text, audio, images, and videos, and using these features for information retrieval, understanding, or generation. Cross-modal interactive fusion aims to achieve more effective data analysis and understanding through joint feature extraction and cross-modal association. For example, by identifying and exploiting the intrinsic connections between different modal data, such as the relationship between image content and the corresponding text description, asymmetric data can be processed, that is, data from one modality may be richer or more detailed than data from another modality.

[0024] The present invention is based on AI technologies such as large language models and cross-modal analysis to automatically generate professional media content from text manuscripts. It also combines the characteristics of audio-visual language to perform a series of processing such as duplicate judgment and length optimization on intelligently matched shots, thereby meeting the requirements of professional media for video production efficiency and quality.

[0025] An intelligent video editing method based on a large language model, such as Figure 1 , Figure 2 As shown, the method steps include: Step 1: Perform cross-modal analysis on the material: When the material is put into the database, AI engines such as cross-modal models and intelligent voice will automatically perform multi-dimensional comprehensive intelligent analysis on the material.

[0026] In order to improve the efficiency of cross-modal intelligent analysis of materials and ensure the accuracy of analysis, this embodiment improves the cross-modal analysis method for images. The steps of the comprehensive intelligent analysis are as follows: Figure 2 As shown, including: Perform transition frame detection on the video, and split the continuous video material into multiple scene segments based on the detection results; For each video scene, key frame extraction is performed; Keyframe cross-modal detection and analysis; Perform cross-modal analysis on key frames, generate vectors and save them to the index library; Perform audio synchronous sound analysis, generate synchronous sound index and save it to the index library; Complete material analysis.

[0027] Transcode the audio and video materials to generate a low-bitrate proxy as the basis for cross-modal analysis to ensure the efficiency of cross-modal material intelligent analysis; detect the transition frame of the video, and split the continuous video material into multiple scene segments based on the detection results; extract key frames for each video scene; in addition to the first frame of the transition frame detection, in order to ensure that the decomposition result does not miss key information, it is also necessary to extract key frames from other picture frames outside the transition frame. Perform cross-modal analysis on the key frames, generate vectors and save them to the index library.

[0028] In the present invention, a method is designed to effectively detect key frame content and ensure that the number of key frames does not increase as much as possible. First, the first frame of the video content is extracted as a key frame every 10 frames, and the key frame after the frame extraction is vector analyzed, and the vector difference between the two key frames before and after is calculated; if the vector difference ∂ between each two consecutive frames is less than the preset value ∆, it is considered that the scene does not need to be further split, and the vector of each analyzed key frame is stored in the index library for preservation; if the difference between the two consecutive key frames before and after is greater than or equal to the preset value ∆, then the middle frame between the two key frames is added, and the sixth frame in the segment is used as the key frame, such as Figure 3 As shown in the video frame grouping diagram, firstly, based on the AI ​​algorithm, the video key frames are intelligently identified; and the video frames between two intelligently identified key frames are grouped into groups of 10 frames each, and the first frame of each group is regenerated into a key frame for cross-modal vector analysis, and the analysis result vector is indexed.

[0029] Through this method, the key content of each frame in a continuously changing scene can be analyzed, avoiding the problem that the key information of continuous pictures may be discarded due to only analyzing the key frames detected by the transition frames.

[0030] Step 2, select the material or material group required for creation from the institutional resource library. When starting the intelligent video editing creation, first select the relevant materials required for this creation from the resource library. You can select one or more related materials for intelligent editing of audio and video content. Selecting materials from the institutional resource library can ensure the authenticity and reliability of the source of the materials and the legality of the copyright. This method uses intelligent editing creation based on institutional resource libraries, local or cloud resource libraries, which can effectively avoid copyright disputes caused by grabbing materials from the Internet, as well as the need to identify the authenticity and reliability of the materials.

[0031] Step 3, importing the text manuscript and rewriting and classifying: the video text manuscript can be directly imported, and the large language model can be used to rewrite the video text manuscript. The general text manuscript may not be suitable for expression through the video lens language. With the help of the large language model, the general text manuscript can be rewritten into a video storyboard script through the semantic understanding of the original text manuscript, and the screen description and narration can be generated. In the technical solution of the present invention, the text content in the storyboard script can also be classified and labeled.

[0032] For [Synchronous Sound], the method of the present invention provides two different marking modes: continuous and single sentence. The continuous mode is suitable for the entire interview or dialogue, and the single sentence is suitable for accurately selecting the whole sentence or phrase spoken from the interview or dialogue. For the text part not marked as [Synchronous Sound], the system defaults to processing it according to the [Main Text] classification.

[0033] Step 4: Automatically use different intelligent matching models to match shots according to different categories of the manuscripts: For the text parts marked as [Synchronous Sound] and [Text] in the previous step, different lens intelligent matching methods are used for matching. For the text content marked as [Synchronous Sound], the system will perform similarity matching on the synchronous sound index based on the semantic understanding of the text manuscript; even if the text manuscript and the text results of speech recognition are not exactly the same, they can be matched according to the results of semantic understanding, which ensures that the written text in the manuscript can be intelligently matched with the spoken language in the material interview and dialogue.

[0034] For the lens result of synchronous sound mode matching, when the lens segment is finally used, the method of the present invention will use the original picture and sound corresponding to the lens to enhance the sense of presence of the final result.

[0035] For the [text] part, the system will match the text with the audio and video content in the vector dimension from the cross-modal index library based on the semantic understanding of the text; based on the matching comparison results, similarity data is formed, and intelligent matching of shots is performed based on the similarity. For the shot results of the text mode matching, when the shot fragment is finally used, only the corresponding picture part of the shot will be used, and the sound part will be replaced by the subsequent speech synthesized audio content.

[0036] Step 5: Adjust the intelligent lens matching results to generate a matching candidate lens group. In the previous step of intelligent lens matching, whether in [Synchronous Sound] mode or [Text] mode, save the matching results of each sentence / paragraph as a lens group and sort them by similarity. Define the maximum number of lenses in the lens group and provide the lens with the highest matching similarity in each lens group as the preferred result for the next step. It is usually recommended to set the maximum number of lenses in the lens group to 10.

[0037] Generally, the present invention will provide the shot with the highest matching similarity in each group of shots (ie, the shot numbered 01) as the preferred result for the next step of processing.

[0038] Step 6, generate a sequence and adjust it according to the audio-visual language model, including intelligently merging scenes according to the time sequence of the previous and next shots in the original material in the shot matching results and using the audio-visual language model to analyze and process the matching shot results.

[0039] In the present invention, in addition to matching and recommending intelligent shots for each sentence / paragraph, the single sentence matching shot result of the previous step is further corrected and adjusted according to the content of the entire manuscript or shot-by-shot script.

[0040] When using the [Synchronous Sound] mode for intelligent shot matching, in addition to using the "text-to-speech" matching based on semantic understanding, post-processing of the shot matching results is also performed for the synchronous sound manuscripts marked as "continuous". That is: for multiple sentences / paragraphs marked as "continuous", the scenes are intelligently merged according to the time sequence of the previous and next shots in the original material in the shot matching results.

[0041] The specific method is: Each shot matching result contains the original material id ClipID, in point IN, out point OUT and other information; Assume that the matching shot results of consecutive sentences / paragraphs of simultaneous interpretation texts are C0, C1, C2... respectively; their corresponding original material IDs are ClipID1, ClipID2, ClipID3... respectively, the corresponding in-points of the original materials are IN1, IN2, IN3... respectively, and the corresponding out-points of the original materials are OUT1, OUT2, OUT3... respectively; First, compare the material information of the two shots C1 and C0 in the first group; Compare whether the corresponding original material IDs of the two shots are the same; If ClipID2 is different from ClipID1, the matching results of the two shots are from different materials, and there is no need to merge the scenes. Proceed to compare the next group of materials C2 and C1; If ClipID2 is the same as ClipID1, it is also necessary to compare the continuity of the two shots. Compare the in-point IN2 of the material of shot C1 with the out-point OUT1 of the material of shot C0; If IN2 - OUT1 < t, where t is a system predefined value, it indicates that the second matching shot result is continuous with the first matching shot result in time, and scene merging is performed; If IN2 - OUT1 ≥ t, where t is a system predefined value, it indicates that the second matching shot result is not continuous with the first matching shot result in time, and no scene merging is performed; And so on until the last matching shot result of the simultaneous interpretation.

[0042] When the [Simultaneous Interpretation] mark in the manuscript ends, this comparison automatically ends; no scene merging analysis is performed between different [Simultaneous Interpretation] paragraphs.

[0043] When using the [Main Text] mode for intelligent shot matching, the intelligent analysis system will calculate and recommend the similarity of independent shot matching for each sentence / paragraph of text; however, for the editing of audio-visual content, it does not process individual sentences, segments, or shots one by one, but needs to consider the whole according to the context.

[0044] When generating a video editing sequence, the present invention innovatively introduces an audio-visual language model to analyze and process the matching shot results. First, duplicate the intelligent matching shot results of each sentence / paragraph of text; starting from the second sentence / paragraph, compare the first shot segment of the current matching candidate shot group with the previous matching result for material duplication; if there is a duplicate, use the next candidate shot in the matching candidate shot group until there is no duplication with the previous shot. If all of the matching candidate shot groups corresponding to this sentence / paragraph of text are duplicates of the previous shot, the matching shot for this sentence / paragraph is left blank, and other shots will be replaced and filled by humans later.

[0045] Secondly, the shot length is processed for the intelligently matched shot results. For shots that are too short, the length of the shot is extended forward or backward until the shot length meets the preset minimum shot length.

[0046] Secondly, the audio-visual speech of the previous and next shots, such as the scene and shooting method, is analyzed to match the montage sentence pattern in the audio-visual language model. If the analysis results of the previous and next shots match any montage sentence pattern, they are retained; otherwise, the current shot is replaced from the matching candidate shot group until it meets the audio-visual language sentence pattern.

[0047] Finally, fine-tune the length of the video shot based on the length of the audio for dubbing or synchronous sound.

[0048] After shot matching for each sentence / paragraph in the manuscript, the shot matching results are arranged and spliced ​​in the order of the manuscript or storyboard script.

[0049] Step 7, generate the soundtrack and subtitles, and add the background music. In the present invention, when performing intelligent editing, the voice recognition and speech synthesis engine can be used to generate the voiceover for the narration part of the editing result, and the subtitles for the main text and the speech in the same period can be automatically generated; the music provided by the system can also be emotionally classified, and the corresponding emotion can be selected to automatically add music to the generated intelligent editing result as needed.

[0050] Step 8, complete intelligent editing and perform manual proofreading to meet the final release review requirements. After the above series of intelligent processing, the intelligent editing timeline result of the method of the present invention has been generated; after the system intelligently generates the timeline result, it can be further proofread by humans to meet the final release review requirements.

[0051] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred solution, a person skilled in the art should understand that the technical solution of the present invention (such as changes in the name or quantity of the categories, the sequence of steps, etc.) can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent video editing method based on a large language model, characterized in that: The method steps include: Step 1: Perform cross-modal analysis on the material: When the material is stored, AI engines such as cross-modal models and intelligent voice will automatically perform multi-dimensional comprehensive intelligent analysis on the material; Step 2: Select the material or material group required for creation from the resource library; Step 3, import the transcript and rewrite and classify it: import the video transcript and use the large language model to rewrite the video transcript into the required transcript, and classify and annotate the synchronous sound and text type; Step 4: Automatically use different intelligent matching models to match shots according to different categories of the manuscripts; Step 5, adjust the intelligent matching results of the shots and generate a matching candidate shot group: save the matching results of each sentence / paragraph as a shot group, sort them by similarity, define the maximum number of shots in the shot group, and provide the shot with the highest matching similarity in each shot group as the preferred result for the next step; Step 6, generating a sequence and adjusting it according to the audio-visual language model, including intelligently merging scenes according to the time sequence of the previous and next shots in the original material in the shot matching results and using the audio-visual language model to analyze and process the matching shot results; Step 7, generate the accompaniment and subtitles, and add music; Step 8: Complete intelligent editing and conduct manual proofreading to meet the review requirements for final release.

2. The intelligent video editing method based on a large language model according to claim 1, characterized in that: The comprehensive intelligent analysis in step 1 includes: 1.1, transcode the audio and video materials to generate a low-bitrate proxy as the basis for cross-modal analysis; 1.2, detect transition frames in the video, and split the continuous video material into multiple scene segments based on the detection results; 1.3, extract key frames for each video scene; 1.4, cross-modal analysis of key frames, generating vectors and saving them to the index library; 1.5, perform audio synchronous sound analysis, generate synchronous sound index and save it to the index library.

3. The intelligent video editing method based on a large language model according to claim 1, characterized in that: In step 4, for text content marked as synchronous sound, the system performs similarity matching on the synchronous sound index based on the semantic understanding of the text manuscript, including the text results of the text manuscript and the speech recognition are not completely the same, so as to ensure intelligent matching between the written text in the manuscript and the spoken language in the material interview and dialogue; for the main text type, the system matches the text with the audio-visual content in the vector dimension from the cross-modal index library based on the semantic understanding of the text, forms similarity data according to the matching comparison results, and performs intelligent matching of the shots according to the similarity.

4. The intelligent video editing method based on a large language model according to claim 1, characterized in that: The method for intelligently merging scenes in step 6 according to the time sequence of the front and back shots in the original material in the shot matching result is as follows: Each shot matching result contains the original material id ClipID, in point IN, out point OUT and other information; Assume that the matching shot results of multiple consecutive sentences / paragraphs of synchronous sound text are C0, C1, C2, ...; their corresponding original material IDs are ClipID1, ClipID2, ClipID3, ..., their corresponding original material entry points are IN1, IN2, IN3, ..., and their corresponding original material exit points are OUT1, OUT2, OUT3, ...; First, compare the material information of the two shots C1 and C0 in the first group to see if the original material IDs corresponding to the two shots are the same; If ClipID2 is different from ClipID1, the two shot matching results come from different materials, and there is no need to merge the scenes. The next set of materials C2 and C1 will be compared. If ClipID2 is the same as ClipID1, you also need to compare the continuity of the two shots, comparing the material in point IN2 of shot C1 with the material out point OUT1 of shot C0; If IN2 —OUT1< t, where t is a system predefined value, it indicates that the second matching shot result is temporally continuous with the first matching shot result, and the scene is merged; If IN2 —OUT1 ≥ t, t is a system predefined value, it means that the second matching shot result is not continuous with the first matching shot result in time, and the scene merging is not performed; And so on, until the last synchronous sound matches the shot result.

5. The intelligent video editing method based on a large language model according to claim 1, characterized in that: The method of using the audio-visual language model to analyze and process the matching shot results in step 6 includes: judging the duplicate shot results of each sentence / paragraph of text; processing the shot length of the intelligently matched shot results; analyzing the audio-visual speech such as the scene type and shooting method of the front and rear shots, and matching them with the montage sentence pattern in the audio-visual language model; fine-tuning the length of the video shot according to the audio length of the dubbing or synchronous sound; after the shot matching of each sentence / paragraph of text in the manuscript, arranging and splicing the shot matching results in the order of the manuscript or the storyboard script.

6. The intelligent video editing method based on a large language model according to claim 1, characterized in that: In step 7, the voice-over part of the editing result can be generated through the speech recognition and speech synthesis engine, and subtitles can be automatically generated for the text and the speech in the same period. The music provided by the system can be classified by emotions, and the corresponding emotions can be selected to automatically add music to the generated intelligent editing result as needed.

7. The intelligent video editing method based on a large language model according to claim 1, characterized in that: The maximum number of lenses is less than or equal to 10.

8. The intelligent video editing method based on a large language model according to claim 2, characterized in that: The step 1.4 of cross-modal analysis of key frames is as follows: extracting the first frame of every 10 frames of the video content as a key frame, performing vector analysis on the key frames after the extraction, and performing difference calculation on the vectors of the two key frames before and after; If the vector difference ∂ between every two consecutive frames is less than the preset value ∆, it is considered that the scene does not need to be further split, and the vector of each key frame analyzed is stored in the index library for preservation; If the difference between two consecutive key frames is greater than or equal to the preset value ∆, the sixth frame in the segment, which is the middle frame between the two key frames, is added as a key frame, and its analysis result vector is indexed.

Images