Video content floc intelligent generation method based on generative artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a video content floc intelligent generation method based on generative artificial intelligence, and the method comprises the steps: S1, data collection: obtaining image-text data from a plurality of sources; s2, data preprocessing: image-text materials are identified, and human faces appearing in a video are clustered; storing a management data preprocessing result; s31, a video performance description fragment data preprocessing method; s32, grouping based on face clustering; s33, sorting and selecting; s34, carrying out ASR and face alignment processing; s35, recording and eliminating repeated character segments; s36, circularly executing fragment selection; s37, carrying out a rollback strategy when a single face is insufficient; s38, carrying out a rollback strategy on a plurality of face fragments; s41, an interactive question and answer fragment data preprocessing method; and S42, determining a point location of an answer fragment. The method has the advantages that the manpower demand is reduced; the production capacity and efficiency as well as the content generation quality and consistency are improved; and multi-mode production is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for intelligently generating video content highlights based on generative artificial intelligence. Background Art

[0002] The current behind-the-scenes video editing process mainly relies on manual processing, including material management, content review and other links. This manual operation often leads to low work efficiency. Although many editing tools on the market provide automated editing functions or preset templates, the editor still needs to make personalized adjustments for complex scenes, emotional expressions and story rhythm. This means that editors need to manually perform editing, color adjustment, dubbing and other tasks, which undoubtedly consumes a lot of energy and time. In addition, different publishing platforms have their own requirements for video resolution, encoding format and duration. Editors have to export different versions of videos for each platform, which undoubtedly increases the workload. Therefore, it is of great practical application value to develop a video content intelligent extraction method based on generative artificial intelligence technology to improve the efficiency of video creation and reduce manual intervention in the fine editing process. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for intelligently generating video content highlights based on generative artificial intelligence, so as to improve the automation of video generation, reduce manual intervention in the fine editing process, and improve work efficiency.

[0004] In order to achieve the above purpose, a method for intelligently generating video content highlights based on generative artificial intelligence is designed, and the method includes S1. data acquisition, obtaining graphic data from multiple sources such as titles, video files, and picture files of video source websites; S2. data preprocessing: for different video clips, the graphic materials are identified by lens, face, behavior, and object through intelligent recognition methods, and the faces appearing in the video are clustered using face clustering data, and the time points at which each character appears in the video are obtained; the full amount of voice text data and corresponding time points of the original video are processed by ASR, and the data preprocessing results are stored and managed; S31. Video performance description clip data preprocessing method, screening out those that meet the conditions Single-person clips, by traversing all clips, select clips containing only a single face, and the clip duration is greater than or equal to the template performance description clip duration; record the timestamps, character IDs, and feature information of these clips to form a preliminary clip list; S32. Based on face clustering grouping: Use an unsupervised clustering algorithm to cluster the faces in the preliminary clip list to ensure that different faces are divided into different groups, and each group represents a different person, so as to control and avoid the same person from appearing repeatedly; S33. Sorting selection: Sort the clip list after clustering grouping, and select a representative clip from the first few clips of each group in the sorted clip list to form a candidate clip list; S34. ASR and face alignment processing :For each candidate segment, process the segment entry and exit points according to the data distribution of its face data and ASR segment; S35. Record and remove duplicate character segments: record the face clustering ID of the selected segment, and remove the segment with this ID from the candidate list to avoid multiple appearances of the same person; S36. Loop segment selection: loop through steps S33, S34, and S35 until 3 segments are selected; S37. Fallback strategy when single face is insufficient: if 3 segments are still not reached, select the segment with a single face but insufficient duration, and sort it by priority and process it by the point alignment logic of step S34; S38. Fallback strategy for multiple face segments: if a single face segment If there are not enough segments, select the segments where multiple faces appear, process them according to the priority logic and point alignment logic, and supplement them to 3 segments; S41. Interactive question and answer segment data preprocessing method: based on the general feature that the questioner will not appear repeatedly, extract the list of segments that appear only once in the entire video and only contain a single face in the picture, and determine the question segment points corresponding to the close-up segments of the questioner; S42. Based on the general feature of the person answering after the question is completed, select the face segments that follow the question segment to determine the answer segment points; S43. If the number of the above selected segments is insufficient, remove the condition of only appearing once in step 1, select the single face segment in the picture as the question segment, and the subsequent segments as the answer segment.

[0005] Preferably, the method of the present invention also includes other technical features, including S0. Configuring the highlights template, and producing three versions of the highlights template according to different video content requirements and modes of the roadshow activities, namely, a basic version, an interactive question and answer video version, and an interactive question and answer text version.

[0006] Preferably, the method of the present invention also includes other technical features, including S5. Generation of highlights video: filling the data preprocessing results into the corresponding video template framework, applying the video editing template, and generating the highlights video.

[0007] Preferably, the method of the present invention also includes other technical features, wherein S33 is specifically as follows: sorting the list of clustered segments, wherein the sorting method satisfies the priority: the earlier the segment is in the video, the higher the priority; the longer the segment duration, the higher the priority.

[0008] Preferably, the method of the present invention also includes other technical features, wherein the S34 is specifically as follows: Complete inclusion situation: If the face segment is completely contained in the ASR segment, the template duration is cut based on the entry point of the face segment; Complete inclusion situation: If the face segment is completely contained by the ASR segment, the template duration is cut based on the entry point of the ASR segment; Partial overlap situation: If there is a partial intersection between the face segment and the ASR segment, a reasonable entry point is selected to ensure that the first frame is a face, while trying to ensure the integrity of the voice content.

[0009] Preferably, the method of the present invention also includes other technical features, wherein S36 is specifically as follows: if the number of selected segments is less than 3, step S33 is executed again in the selected face cluster IDs to try to supplement from the unselected segments.

[0010] Compared with the prior art, the present invention has the following advantages: 1. Cost control: by reducing manpower requirements, the impact of differences in personal experience, preferences and aesthetics can be reduced; 2. Production efficiency: With the help of artificial intelligence technology, the capacity and efficiency of content production have been significantly improved; 3. Quality consistency: The quality and consistency of content generation are continuously improved; 4. Multimodal production: realize cross-modal production of different forms of content such as pictures, texts, and videos, enriching the means of content production; 5. Operational benefits: By activating content, enhancing user experience, and improving the business effect of "short videos driving long videos", the value of content operations can be increased. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, principle and structure of the present invention more clear, it is further described below in conjunction with the drawings and specific embodiments.

[0013] The purpose of the present invention is to provide a method for intelligently generating video content highlights based on generative artificial intelligence. The method can extract face clustering and ASR voice data of the original video of a corporate performance briefing through an intelligent recognition function, select segments based on the recognition data, and finally generate a highlights video in combination with a customized template.

[0014] Embodiment 1: In this embodiment, in order to realize the above technical solution, a specific implementation method is selected.

[0015] 1. Configure the behind-the-scenes template: According to the different video content requirements and modes of the roadshow activities, three versions of the behind-the-scenes template are produced. They are the basic version, the interactive question and answer video version, and the interactive question and answer text version. a) Basic version: Content includes the event title, company text introduction, company picture introduction, video performance introduction, and ending summary; b) Interactive question and answer video version: Content includes the event title, company text introduction, company picture introduction, video performance introduction, interactive question and answer screen introduction, and ending summary; c) Interactive question and answer text version: Content includes the event title, company text introduction, company picture introduction, video performance introduction, interactive question and answer text introduction, and ending summary; 2. Data collection (acquisition of graphic and text content): Establish a link between the highlight generation website and the SSE Roadshow Center website (i.e., the video source website), build a data collection module, obtain graphic and text data from multiple sources such as the roadshow website title, video files, and image files in the form of a custom development interface, and organize the video and graphic content according to the scene category.

[0016] 3. Data preprocessing: For different clips such as performance briefings and interactive question-and-answer videos, the intelligent recognition function is used to identify shots, faces, behaviors, objects, etc. of graphic materials. Face clustering data is used to cluster the faces that appear in the video, and the time points at which each character appears in the video are obtained; ASR is used to store and manage the data preprocessing results of the full amount of voice text data and corresponding time points of the original video.

[0017] The data preprocessing logic is as follows: 1) Performance description segment data preprocessing process: Step 1: Filter eligible single-person clips: traverse all clips and select clips that only contain a single face, and the clip length is greater than or equal to the template performance description clip length; record the timestamps, person IDs, features and other information of these clips to form a preliminary clip list.

[0018] Step 2: Grouping based on face clustering: Use an unsupervised clustering algorithm (such as K-Means, DBSCAN) to cluster the faces in the preliminary clip list to ensure that different faces are divided into different groups to distinguish different leaders; each group represents a different person (i.e., a different leader), thereby controlling to avoid repeated appearances of the same leader.

[0019] Step 3: Sorting and selection: Sort the clustered segment list to meet the following priorities: 1. The earlier the segment is in the video, the higher the priority; 2. The longer the segment is, the higher the priority; for the sorted segment list, select a representative segment from the first few segments in each group to form a "candidate segment" list.

[0020] Step 4: ASR and face alignment processing: For each candidate segment, process the segment entry and exit points according to the distribution of its face data and the ASR segment data to ensure the synchronization of the picture and voice: 1). Complete inclusion: If the face segment is completely contained in the ASR segment, the template duration is cut based on the entry point of the face segment. 2). Complete inclusion: If the face segment is completely contained in the ASR segment, the template duration is cut based on the entry point of the ASR segment. 3). Partial overlap: If the face segment and the ASR segment have partial intersections, select a reasonable entry point to ensure that the first frame is a face, while trying to ensure the integrity of the voice content.

[0021] Step 5: Record and remove duplicate leader segments: Record the face cluster ID of the selected segment and remove the segment with this ID from the candidate list to avoid the same leader appearing multiple times.

[0022] Step 6: Loop through the segment selection process: Loop through steps 3, 4, and 5 until three segments are selected. If fewer than three segments are selected, execute step 3 again in the selected face cluster IDs and try to add more segments from the unselected segments.

[0023] Step 7: Fallback strategy when there are not enough single faces: If the number of segments is still less than 3, select the segments where a single face appears but the duration is insufficient, sort them by priority and process them according to the point alignment logic of step 4.

[0024] Step 8: Fallback strategy for multiple face segments: If there are insufficient single face segments, select segments where multiple faces appear, process them according to the priority logic and point alignment logic (same as step 4), and supplement them to 3 segments.

[0025] 2) Interactive question and answer segment data preprocessing process: Step 1: Based on the common feature that the questioner will not appear repeatedly, extract the list of segments that appear only once in the entire video and contain only a single face, corresponding to the close-up segments of the questioner, and determine the question segment points; Step 2: Based on the general characteristics of the leader's answer after the question is completed, select the face segment following the question segment and determine the answer segment point; Step 3: If the number of selected segments is insufficient, remove the condition that only appears once in step 1, select a single face segment in the picture as the question segment, and the subsequent segments as the answer segment; 4. Behind-the-scenes video generation: Fill the data preprocessing results into the corresponding video template framework, apply the video editing template, and generate behind-the-scenes videos. Manually adjust the self-generated behind-the-scenes videos in the built-in quick editing interface, including text introductions, picture introductions, text interactive Q&A, performance introduction clips, video interactive clips, etc.

[0026] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes according to the technical solutions and novel concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for intelligently generating video content highlights based on generative artificial intelligence, characterized in that The method comprises S1. Data collection, obtaining graphic data from multiple sources such as titles, video files, and picture files of video source websites; S2. Data preprocessing: For different segments of the video, the image and text materials are identified by lens, face, behavior, and object through intelligent recognition methods. The faces appearing in the video are clustered using face clustering data, and the time points at which each person appears in the video are obtained. The full amount of voice text data and corresponding time points of the original video are processed through ASR, and the data preprocessing results are stored and managed. S31. Video performance description segment data preprocessing method, screening qualified single-person segments, by traversing all segments, selecting segments containing only a single face, and the segment duration is greater than or equal to the template performance description segment duration; recording the timestamps, character IDs, and feature information of these segments to form a preliminary segment list; S32. Grouping based on face clustering: using an unsupervised clustering algorithm to cluster faces in the preliminary clip list, ensuring that different faces are grouped in different groups, each group representing a different person, so as to avoid repeated appearance of the same person; S33. Sorting selection: sorting the fragment list after clustering and grouping, and selecting a representative fragment from the first few fragments of each group to form a candidate fragment list; S34. ASR and face alignment processing: for each candidate segment, process the segment entry point and exit point according to the data distribution of its face data and the ASR segment; S35. Record and remove duplicate character segments: record the face cluster ID of the selected segment, and remove the segment with this ID from the candidate list to avoid multiple appearances of the same person; S36. Loop through the fragment selection: Loop through steps S33, S34, and S35 until three fragments are selected; S37. Fallback strategy when single face is insufficient: If the number of segments is still less than 3, select the segments where a single face appears but the duration is insufficient, and sort them by priority and process the point alignment logic of step S34; S38. Fallback strategy for multiple face segments: If the single face segment is insufficient, select the segments where multiple faces appear, process them according to the priority logic and point alignment logic, and add them to 3 segments; S41. Interactive question and answer segment data preprocessing method: Based on the general feature that the questioner will not appear repeatedly, extract the segment list that appears only once in the entire video and contains only a single face, corresponding to the close-up segment of the questioner, and determine the question segment point; S42. Based on the common characteristics of the person answering after the question is completed, select the face segment following the question segment and determine the answer segment point; S43. If the number of selected segments is insufficient, remove the condition that only appears once in step 1, select a single face segment in the picture as the question segment, and the subsequent segments as the answer segments.

2. The method for intelligently generating video content highlights based on generative artificial intelligence as claimed in claim 1, characterized in that Also includes S0. Configure the behind-the-scenes template. According to the different video content requirements and modes of the roadshow activities, three versions of the behind-the-scenes template are produced, namely the basic version, the interactive Q&A video version, and the interactive Q&A text version.

3. The method for intelligently generating video content highlights based on generative artificial intelligence as claimed in claim 2, characterized in that Also includes S5. Generation of highlights video: Fill the data preprocessing results into the corresponding video template framework, apply the video editing template, and generate the highlights video.

4. The method for intelligently generating video content highlights based on generative artificial intelligence as claimed in claim 1, characterized in that The S33 is specifically as follows: sorting the segment list after clustering and grouping, and the sorting method satisfies the priority: the earlier the position of the segment in the video, the higher the priority; the longer the segment duration, the higher the priority.

5. The method for intelligently generating video content highlights based on generative artificial intelligence as claimed in claim 1, characterized in that The S34 is specifically as follows: Complete inclusion: If the face segment is completely included in the ASR segment, the template duration is cut based on the entry point of the face segment; Complete inclusion: If the face segment is completely included in the ASR segment, the template duration is cut based on the entry point of the ASR segment; Partial overlap: If there is a partial intersection between the face segment and the ASR segment, a reasonable entry point is selected to ensure that the first frame is a face, while trying to ensure the integrity of the voice content.

6. The method for intelligently generating video content highlights based on generative artificial intelligence as claimed in claim 1, characterized in that The specific steps of S36 are as follows: if the number of selected segments is less than 3, then step S33 is executed again in the selected face cluster IDs to try to supplement from the unselected segments.

Images