Intelligent fine-cutting method and system for interview long material

Abstract

The present application relates to an intelligent fine cutting method and system for long interview materials. The method comprises: separating audio and video materials into audio stream and video stream; performing voice enhancement, speech detection and clustering on the audio stream, and labeling the speakers, overlapping speech and silence sections; performing automatic multi-language speech recognition, and forcibly aligning and correcting the word timestamp with the timestamp in the video stream to generate a candidate cut point set and smoothness score; using a large language model to understand user intent, extracting text content and mapping it to word timestamp, adding context buffer and alternative candidate segments to each mapping, and recording mapping confidence; scoring candidate segments based on a multi-dimensional scoring function, selecting a segment sequence within the target time length, and generating a standard timeline file according to the segment sequence, and outputting a reference film and subtitle preview. The method can shorten the period of manual screening and repeated trial cutting, and improve the quality and consistency of the film.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an intelligent editing method and system for long interview materials. Background Technology

[0002] With the rapid growth of news documentaries, brand communication, podcasts, and new media content, interview and interview footage has become one of the most common forms of long-form video / audio content. This type of footage typically ranges in length from tens of minutes to several hours, while the final product is often strictly limited to tens of seconds to several minutes. Editors need to balance information density, emotional pacing, and audio-visual quality while ensuring thematic focus and narrative coherence. Furthermore, the same footage is often repeatedly used in multiple platforms and versions, placing higher demands on the automation, traceability, and standardized output of the editing process. Currently, the mainstream industry approach still heavily relies on manual or semi-manual operations by editors, involving "overview-transcription-positioning-trimming-sponging." Specifically, editors first review, transcribe, and roughly annotate the long footage, then use a non-linear editing system to locate cut points sentence by sentence and repeatedly test edits to meet the thematic expression and target length requirements. This process is time-consuming, labor-intensive, highly dependent on the experience of practitioners, and prone to issues such as strong subjectivity, inconsistent tastes, and high iteration costs. The presence of multiple cameras, multiple speakers, accent differences, environmental noise, and overlapping speech further increases the difficulty of achieving high transcription accuracy and stable cut-off points.

[0003] Several single-point auxiliary tools have emerged in existing technologies, such as Automatic Speech Recognition (ASR), keyword retrieval, hotspot sentence extraction, or simple shot optimization. However, most of these solutions lack end-to-end linkage capabilities under the dual constraints of "topic and target duration." Specifically, their shortcomings are reflected in the following aspects: Speech side: ASR accuracy and sentence segmentation quality decrease in accented, multilingual, and noisy environments; speaker diarization is unstable with overlapping speech and rapid transitions; long-term processing easily leads to timestamp drift, resulting in accumulated alignment errors. Semantic side: Only keywords or summaries are available, lacking reliable mapping to original sentence / word-level timestamps, making it difficult to achieve bidirectional traceability and interpretability between "summary-material"; there is a lack of overall control over narrative structure and semantic deduplication. Editing side: Cutting points often rely on simple heuristics such as waveform pauses, ignoring lip closure, breathing, and semantic integrity, easily resulting in "word truncation," "cutting in," and "abrupt rhythm"; in multi-camera scenarios, there is a lack of a joint optimization strategy based on image quality, gaze direction, facial expression clarity, and shot stability. On the optimization side: There's a lack of a scoring function and combined optimization framework that unifies multi-dimensional indicators such as topic relevance, information density, coherence, role balance, emotional curves, audio-visual quality, and compliance into adjustable weights, making it difficult to automatically fit globally optimal or suboptimal results within the target duration. On the interoperability side: There's insufficient deep interoperability with mainstream NLEs in the industry, lacking standardized EDL / FCPXML / AAF timeline export and rich metadata injection, making it difficult to efficiently refine and continue automatic results. On the quality and compliance side: There's a lack of audio-visual quality assessment, redundancy / slip-of-speech identification, sensitive content and legal risk detection, and alternative suggestions for industrial delivery, making it difficult to meet review and release requirements.

[0004] Therefore, traditional methods of editing footage rely heavily on manual or semi-manual operations, and existing single-point technologies can only achieve fragmented capabilities such as voice recognition and keyword retrieval. They lack end-to-end collaboration under the dual constraints of theme and duration, often resulting in low efficiency and strong subjectivity, making it difficult to meet review and compliance requirements. Summary of the Invention

[0005] Based on this, in order to solve the above-mentioned technical problems, an intelligent fine-tuning method and system for interview-type long footage is provided, which can shorten the cycle of manual screening and repeated trial editing, and improve the quality and consistency of the final product.

[0006] A method for intelligent editing of long interview clips, the method comprising:

[0007] The imported audio and video materials are obtained, the audio stream and video stream are separated, and the temporal base of the audio stream and video stream are kept consistent.

[0008] The audio stream is enhanced with human voice, and speech activity is detected and clustered. Each voiceprint feature in the audio stream is obtained. The speaker is labeled with the voiceprint features, and overlapping speech and silent segments are also labeled.

[0009] The audio stream is subjected to multilingual automatic speech recognition to generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores. The word and sentence timestamps are then forcibly aligned and corrected with the timestamps in the video stream to generate a candidate cut point set and a smoothness score.

[0010] Collect user intent data, use a large language model to perform semantic understanding on the user intent data, extract text content and map it to the word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence.

[0011] The candidate segments are scored using a multidimensional scoring function. Based on the scoring results, a segment sequence is selected within the target duration based on the candidate cut point set and smoothness score. A standard timeline file is generated based on the segment sequence, and a reference finished product and subtitle preview are output.

[0012] In one embodiment, acquiring imported audio and video materials, separating the audio stream and video stream, and maintaining the temporal base consistency of the audio stream and video stream includes:

[0013] Obtain the imported audio and video materials, and register the material identifiers in the audio and video materials;

[0014] The audio and video materials are decapsulated to separate the audio stream and video stream, and the temporal base consistency of the audio stream and video stream is maintained based on the material identifier.

[0015] In one embodiment, the audio stream is subjected to voice enhancement, speech activity detection, and clustering, and various voiceprint features in the audio stream are obtained. Speaker roles are then labeled based on these voiceprint features, and overlapping speech and silent segments are also labeled, including:

[0016] The audio stream is subjected to a short-time Fourier transform, a voice mask is generated by a deep neural network, and the enhanced voice audio stream is recovered by inverse transform to achieve voice enhancement.

[0017] The audio stream after voice enhancement is framed, acoustic features are extracted, and it is determined whether each frame is valid speech, and speech segments and non-speech segments are divided.

[0018] Voiceprint features are extracted from each of the speech segments, and speaker clusters are performed based on the voiceprint features. A unique speaker identifier is assigned to each cluster.

[0019] Based on the voiceprint features, the speaker identifier is mapped to a specific role, thus completing the role labeling;

[0020] Overlap detection is performed on the speech segments of multiple speakers, overlapping speech segments are identified and marked; the non-speech segments are marked as silent segments.

[0021] In one embodiment, multilingual automatic speech recognition is performed on the audio stream to generate a recognition result containing punctuation marks, word and phrase timestamps, and confidence levels, including:

[0022] The audio stream is subjected to language recognition to obtain the language category. Based on the language category, a multilingual automatic speech recognition model is determined to extract features, and the recognized text is generated by the decoder.

[0023] The pause and intonation information extracted from the multilingual automatic speech recognition model are combined to add punctuation marks to the recognized text;

[0024] The text containing the punctuation marks is segmented into independent sentence units, and the start and end timestamps of each sentence unit are recorded.

[0025] Record the acoustic frame time position corresponding to each word, and generate word-level start and end timestamps;

[0026] Calculate the decoding matching probability of each word and phrase, and generate word-phrase level confidence scores.

[0027] In one embodiment, the timestamps of the words and phrases are forcibly aligned and corrected with the timestamps in the video stream to generate a candidate cut-off point set and a smoothness score, including:

[0028] Based on the unified time base of the audio and video materials, an initial mapping relationship is established between the word timestamps and the timestamps in the video stream;

[0029] The audio and video materials are divided into various detection windows. The drift deviation value between the word and phrase timestamps and the timestamps in the video stream is calculated, and the word and phrase timestamps are forcibly corrected based on the drift deviation value.

[0030] Based on the corrected word and sentence timestamps, sentence boundaries, word boundaries, silent segment boundaries, overlapping speech boundaries, and pause points are collected as candidate cut point sets;

[0031] Based on the dimensions of semantic integrity, speech continuity, energy smoothness, mouth closure degree, and rhythmic adaptability, the smoothness score of each candidate cut point in the candidate cut point set is calculated.

[0032] In one embodiment, user intent data is collected, and a large language model is used to perform semantic understanding on the user intent data. Text content is extracted and mapped to word and sentence timestamps. Contextual buffers and alternative candidate segments are added to each mapping, and mapping confidence is recorded, including:

[0033] Collect user input including topic descriptions, target duration, style preferences, role orientations, and compliance constraints to form structured user intent data;

[0034] The user intent data and multilingual automatic speech recognition results are input into a large language model. The semantic parsing and similarity matching algorithm of the large language model are used to complete the semantic understanding of the user intent and extract the text content based on the semantic understanding.

[0035] The text content is mapped to the word and phrase timestamps in the multilingual automatic speech recognition results to establish a text-time mapping table;

[0036] For each mapping relationship in the mapping table, a context buffer space of a preset market is set, and alternative candidate segments are retrieved from the audio and video materials;

[0037] Generate the mapping confidence score for each of the mapping relationships, and record the context buffer, alternative candidate segments, and mapping confidence scores into the mapping relationship table.

[0038] In one embodiment, the candidate segments are scored based on a multidimensional scoring function. Based on the scoring results, and using the candidate cut-off point set and smoothness score, a segment sequence is selected within a target duration. A standard timeline file is then generated based on the segment sequence, including:

[0039] Construct a multidimensional scoring function and configure adjustable weights for the multidimensional scoring function. Use the multidimensional scoring function to quantify and score the candidate segments and sort them from high to low scores.

[0040] Using the user intent data as a constraint, a sequence of segments is selected from the sorted candidate segments using a combination of greedy algorithm and dynamic programming.

[0041] The segment sequence is fine-tuned at the cut points, and the selected segments are arranged sequentially on the video track and audio track. The recognized text corresponding to the selected segments is used to generate a subtitle track, and tags are written into the timeline to generate a standard timeline file.

[0042] In one embodiment, selecting a sequence of fragments from the sorted candidate fragments includes:

[0043] The context buffer is trimmed from the timed-out segments in the sorted candidate segments, and candidate segments are added to segments with insufficient duration or adjacent semantically coherent segments are merged to obtain the segment sequence.

[0044] In one embodiment, the method further includes:

[0045] Based on the standard timeline file, the audio and video materials are decoded and re-encoded to output a low-bitrate reference video and an embedded subtitle video, enabling a synchronized preview of the reference video and the subtitles.

[0046] A smart editing system for long interview clips, the system comprising:

[0047] The material acquisition and processing module is used to acquire imported audio and video materials, separate the audio stream and video stream, and maintain the consistency of the time base of the audio stream and video stream;

[0048] The audio processing module is used to enhance human voice, detect and cluster speech activity in the audio stream, and obtain various voiceprint features in the audio stream. Based on the voiceprint features, the speaker is labeled with a role, and overlapping speech and silent segments are also labeled.

[0049] The audio recognition and alignment module is used to perform multilingual automatic speech recognition on the audio stream, generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores, and forcibly align and correct the word and sentence timestamps with the timestamps in the video stream to generate a candidate cut point set and a smoothness score.

[0050] The text and time mapping module is used to collect user intent data, perform semantic understanding on the user intent data using a large language model, extract text content and map it to the word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence.

[0051] The timeline generation and export module is used to score the candidate segments based on a multi-dimensional scoring function. Based on the scoring results, the module selects a segment sequence within the target duration based on the candidate cut point set and smoothness score, generates a standard timeline file based on the segment sequence, and outputs a reference finished product and subtitle preview.

[0052] The aforementioned intelligent editing method and system for long interview clips improves the stability and accuracy of speech processing in complex audio environments by maintaining audio-video time base consistency and performing voice enhancement, speaker role labeling and overlapping speech, and silent segment marking. Through multilingual speech recognition and forced alignment correction of audio-video timestamps, it effectively eliminates drift errors in long-duration clips, obtaining natural and reliable candidate cut points. By understanding user intent through a large language model and establishing a bidirectional mapping between text and time, it ensures focused editing, semantic integrity, and replaceable optimization. By selecting segment sequences through multi-dimensional scoring, it achieves automated editing with focused themes, coherent narrative, and precise duration. Finally, it outputs a standard timeline, a reference cut, and a subtitle preview, significantly reducing manual editing costs and improving the quality and consistency of the final cut. Attached Figure Description

[0053] Figure 1 This is an application environment diagram of an intelligent fine-tuning method for long interview materials in one embodiment;

[0054] Figure 2 This is a flowchart illustrating an intelligent editing method for long interview materials in one embodiment.

[0055] Figure 3 This is a schematic diagram of the text-time mapping relationship in one embodiment;

[0056] Figure 4 This is a flowchart illustrating the intelligent editing method for long interview materials in another embodiment;

[0057] Figure 5 This is a structural block diagram of an intelligent editing system for long interview materials in one embodiment.

[0058] Figure 6 This is a structural block diagram of an intelligent editing system for long interview materials in another embodiment;

[0059] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] The intelligent editing method for long interview materials provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the application environment includes computer device 110. Computer device 110 can acquire imported audio and video materials, separate the audio stream and video stream, and maintain temporal consistency between the audio and video streams; computer device 110 can perform voice enhancement, speech activity detection and clustering on the audio stream, and acquire various voiceprint features in the audio stream, assigning roles to speakers based on the voiceprint features, and simultaneously labeling overlapping speech and silent segments; computer device 110 can perform multilingual automatic speech recognition on the audio stream, generating recognition results containing punctuation marks, word and sentence timestamps, and confidence levels, and forcibly matching the word and sentence timestamps with the timestamps in the video stream. The process involves alignment and correction, generating a candidate cut-off point set and a smoothness score. Computer device 110 can collect user intent data, use a large language model to perform semantic understanding of the user intent data, extract text content and map it to word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence. Computer device 110 can score candidate segments based on a multi-dimensional scoring function, and based on the scoring results, select a segment sequence within the target duration based on the candidate cut-off point set and smoothness score, generate a standard timeline file based on the segment sequence, and output a reference finished product and subtitle preview. Computer device 110 can be, but is not limited to, various personal computers, laptops, smartphones, robots, tablets, and other devices.

[0062] In one embodiment, such as Figure 2 As shown, an intelligent editing method for long interview clips is provided, including the following steps:

[0063] Step 202: Obtain the imported audio and video materials, separate the audio stream and video stream, and maintain the consistency of the time base of the audio stream and video stream.

[0064] First, computer equipment can perform material import and metadata registration steps, which mainly include import, deduplication, metadata registration, indexing and caching.

[0065] In one embodiment, the intelligent editing method for long interview materials may further include a process of separating audio and video streams. The specific process includes: acquiring imported audio and video materials and registering material identifiers in the audio and video materials; decapsulating the audio and video materials to separate the audio stream and video stream, and maintaining the temporal consistency of the audio stream and video stream based on the material identifiers.

[0066] When importing materials and registering metadata, computer equipment supports video / audio, multi-camera setups, and registering material identifiers such as timecode, frame rate, and camera position. Next, the computer equipment can decapsulate the acquired audio and video materials and maintain time base consistency, separating the audio and video streams.

[0067] Step 204: Perform voice enhancement, speech activity detection and clustering on the audio stream, and obtain each voiceprint feature in the audio stream. Mark the speaker's role based on the voiceprint features, and mark overlapping speech and silent segments.

[0068] Computer devices can separate sound sources from speakers and perform voice enhancement, speech activity detection (VAD), speaker detection and clustering on audio streams. They can also label roles such as hosts and guests by combining voiceprints and language features, and can label overlapping speech and silent segments.

[0069] In one embodiment, the intelligent editing method for long interview materials may further include a speech recognition process, specifically including: performing a short-time Fourier transform on the audio stream, generating a voice mask through a deep neural network, and recovering the enhanced voice audio stream through an inverse transform to achieve voice enhancement; segmenting the enhanced voice audio stream into frames, extracting acoustic features, and determining whether each frame is valid speech, dividing it into speech segments and non-speech segments; extracting voiceprint features from each speech segment, clustering speakers based on the voiceprint features, and assigning a unique speaker identifier to each cluster; mapping the speaker identifier to a specific role based on the voiceprint features, completing role labeling; performing overlap detection on the speech segments of multiple speakers, identifying and marking overlapping speech segments; and marking non-speech segments as silent segments.

[0070] The computer device can first perform voice enhancement processing on the input audio stream, improving the voice signal-to-noise ratio through noise reduction, echo suppression, reverberation suppression, and gain equalization. Specifically, the computer device can perform a short-time Fourier transform on the separated audio stream, generate a voice mask through a deep neural network to suppress non-voice components, and recover the enhanced voice audio stream through an inverse transform. Next, the computer device can perform speech activity detection (VAD) on the enhanced audio. By segmenting the enhanced voice audio stream into frames, extracting acoustic features, and using a speech activity detection classifier to determine whether each frame is valid speech, the computer can merge consecutive speech frames into speech activity segments and output silent segments to obtain the preliminary speech time interval.

[0071] Next, the computer device can extract voiceprint features from the effective speech segments, perform speaker clustering based on the voiceprint features, group speech segments belonging to the same speaker into one category, forming several independent speaker channels, assign a unique speaker identifier to each cluster, and then label each speaker channel as host, guest or other roles according to the clustering results and preset role rules. At the same time, time domain overlap judgment is performed on multiple speaker channels, overlapping speech segments are identified and marked, and the no-energy, low-energy or pure environmental noise intervals output by VAD are marked as silent segments, thus obtaining speech segmentation results with speaker ID, role label, overlap mark and silence mark.

[0072] Step 206: Perform multilingual automatic speech recognition on the audio stream to generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores. Force the alignment and correction of word and sentence timestamps with the timestamps in the video stream to generate a candidate cut point set and smoothness score.

[0073] Computer equipment can perform automatic speech recognition and sentence segmentation, mainly through multilingual automatic speech recognition (ASR), generating punctuation, word / sentence-level timestamps, and confidence scores.

[0074] In one embodiment, the intelligent editing method for long interview materials may further include speech recognition and sentence segmentation processes. Specifically, the process includes: performing language recognition on the audio stream to obtain the language category; determining a multilingual automatic speech recognition model based on the language category for feature extraction; generating recognized text through a decoder; combining the pause and intonation information extracted by the multilingual automatic speech recognition model to add punctuation marks to the recognized text; segmenting the punctuated text into independent sentence units and recording the start and end timestamps of each sentence unit; recording the acoustic frame time position corresponding to each word and generating word-level start and end timestamps; and calculating the decoding matching probability of each word and sentence to generate word-level confidence.

[0075] Computer devices can extract acoustic features from audio streams and input them into a language recognition module to determine the main language category of the audio, such as Chinese, English, Japanese, or mixed languages. Based on the language category recognition results, the corresponding multilingual automatic speech recognition model is selected from a multilingual model library.

[0076] Next, the computer device can use a multilingual automatic speech recognition model to segment the audio stream into frames of fixed length and extract acoustic features. Specifically, the computer device can input the acoustic feature sequence into the multilingual automatic speech recognition model, calculate the recognition probability of phonemes or characters / words frame by frame, and generate the optimal text hypothesis sequence through a decoder under the constraints of a given language model, outputting the original recognized text without punctuation or sentence segmentation. Then, the computer device can combine data such as pauses, intonation, and energy changes from the acoustic features, as well as grammatical structure information from the language model, to predict the type of punctuation mark to be inserted after each word / character and add punctuation marks, outputting the complete text with punctuation marks. Then, the computer device can use terminating punctuation marks such as periods, question marks, exclamation marks, and ellipses as the main sentence boundaries, combined with internal punctuation marks such as commas and semicolons for auxiliary judgment, to divide the text into independent sentence units. At the same time, it can refer to the length of silent segments in the audio. If the silent segment exceeds the threshold, it can be forcibly broken into a sentence. If the silent segment is short but semantically complete, it can also be broken into a sentence. For each segmented sentence, its start timestamp and end timestamp in the audio are recorded. Finally, the output sentence contains the text content, start timestamp, and end timestamp.

[0077] In the speech recognition process of this embodiment, the computer device can record the time position of the acoustic frame corresponding to each word / character. Through forced alignment or forward-backward algorithm, the recognized word / character sequence is matched with the audio waveform, and a start timestamp and an end timestamp are generated for each word / character. Finally, a list of word / character-level timestamps containing text, start timestamp, and end timestamp is output.

[0078] Computer equipment can calculate the credibility of the word / character recognition result based on the probability distribution output by the multilingual automatic speech recognition model during the decoding process, and use it as the word / character level confidence score. It can also obtain the sentence level confidence score from the average, minimum or weighted average of the confidence scores of each word / character in the sentence, and thus output word / character level confidence score labels and sentence level confidence score labels.

[0079] In one embodiment, the intelligent editing method for long interview materials may further include a forced alignment and correction process. Specifically, this process includes: establishing an initial mapping relationship between word / sentence timestamps and timestamps in the video stream, based on a unified time base of the audio / video materials; dividing the audio / video materials into detection windows, calculating the drift deviation value between word / sentence timestamps and timestamps in the video stream, and forcibly correcting the word / sentence timestamps based on the drift deviation value; collecting sentence boundaries, word boundaries, silent segment boundaries, overlapping speech boundaries, and pause points as candidate cut-off point sets based on the corrected word / sentence timestamps; and calculating the smoothness score of each candidate cut-off point in the candidate cut-off point set according to dimensions of semantic integrity, speech continuity, energy smoothness, lip closure, and rhythm adaptability.

[0080] Computer equipment can correct long-term drift and boundary errors, and generate candidate cut points and smoothness scores. Specifically, to ensure that audio timestamps and video timestamps use the same reference system and eliminate benchmark differences caused by audio-video separation or different encoding formats, the computer equipment can read the time base of the audio stream and the time base of the video stream, and uniformly convert the audio timestamps into a time base consistent with the video timecode, thereby establishing an initial mapping relationship between the word / phrase timestamps and the timestamps in the video stream. Next, the computer equipment can divide the entire clip into multiple detection windows of fixed duration. Within each window, several anchor points are selected, and the audio timestamps of the anchor points are compared with the corresponding video frame timestamps. The offset at each anchor point is calculated, and then the trend of the offset as the clip duration increases is used to determine whether drift exists, and the drift start position, drift end position, drift type, and drift amplitude are recorded.

[0081] Next, the computer device can systematically correct the detected drift, realigning the audio and video timestamps and eliminating boundary errors. Specifically, the computer device can maintain the relative order and spacing of word / character-level and sentence-level timestamps, ensuring the correction amount does not cause timestamp reversal, and the corrected timestamps fall within the effective temporal range of the video frames, thus outputting the corrected word / sentence-level timestamps. Using the corrected timestamps as initial boundaries, the computer device can, within a small window near the boundaries, combine audio waveform energy and speech activity detection results to reposition a more precise boundary; then, using the short-time energy envelope of the audio, it can identify the start and end points of the speech, outputting refined word / sentence-level timestamps and recording the boundary adjustment amount.

[0082] Finally, the computer device can generate a set of candidate cut point positions based on the corrected and aligned timestamps, which can be used for subsequent editing decisions. It then evaluates the smoothness of the transition between each candidate cut point and adjacent cut points, providing a quantitative indicator of cut point quality for subsequent editing decisions. Specifically, for each candidate cut point, the computer device can calculate a single-point smoothness score by integrating various dimensions, and for the intervals between adjacent cut points, it can calculate an interval smoothness score. The scores can be weighted and the weights of each dimension can be configured.

[0083] Step 208: Collect user intent data, use a large language model to perform semantic understanding on the user intent data, extract the text content and map it to word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence.

[0084] Users can input intent description information such as topic, target duration, style / pacing preferences, role inclination, and compliance constraints through computer devices, providing constraints and priorities for subsequent mapping.

[0085] In one embodiment, the intelligent editing method for long interview materials may further include a process of user intent understanding and extraction. The specific process includes: collecting user-inputted topic descriptions, target durations, style preferences, role orientations, and compliance constraints to form structured user intent data; inputting the user intent data and multilingual automatic speech recognition results into a large language model, using the large language model's semantic parsing and similarity matching algorithms to complete the semantic understanding of the user intent, and extracting text content based on the semantic understanding; mapping the text content to word and sentence timestamps in the multilingual automatic speech recognition results to establish a text-time mapping table; setting a preset market context buffer for each mapping relationship in the mapping table, and retrieving alternative candidate segments from the audio and video materials; generating a mapping confidence score for each mapping relationship, and recording the context buffer, alternative candidate segments, and mapping confidence scores in the mapping table.

[0086] like Figure 3 As shown, the computer device can match user-input keywords with multilingual automatic speech recognition results, filter out key points related to the topic, and adjust the priority of key points according to the user's style / rhythm preferences, thereby outputting a weighted list of key points that incorporates the user's intent. Next, for each summary key point, a search is performed in the ASR-recognized text using keyword matching, semantic similarity matching, and logical position matching. After a successful match, the start and end timestamps of the sentence are extracted, and finally, a text-time mapping table containing key point text, sentence text, start time, end time, matching method, and matching score is output.

[0087] Next, the computer device can add additional time buffers before and after each mapped segment to ensure that the edited segments are semantically complete and have natural transitions. Each mapped segment with context buffering includes information such as the original start / end time, the buffered start / end time, and the buffering direction. In this embodiment, as... Figure 3 As shown, the computer device can also generate multiple candidate segments for each mapping point, allowing for the selection of the optimal combination during subsequent duration fitting. Each candidate in the generated list of alternative candidate segments includes a timestamp, source type, and similarity to the original point. The computer device can evaluate the confidence level of each mapping result, outputting the overall confidence level and scores for each dimension, as well as review prompts for low-confidence mappings, for subsequent editing decisions and supporting manual review of low-confidence mappings. Finally, the computer device can record the context buffer, alternative candidate segments, and overall mapping confidence level in a text-time mapping relationship table, forming a traceable and replaceable structured mapping result.

[0088] Step 210: Score the candidate segments based on the multidimensional scoring function. Based on the scoring results, select the segment sequence within the target duration based on the candidate cut point set and smoothness score, generate a standard timeline file based on the segment sequence, and output the reference finished film and subtitle preview.

[0089] Computer equipment can perform combination optimization and micro-pruning within the target duration based on the scoring function, ensuring structural integrity.

[0090] In one embodiment, the intelligent fine-tuning method for long interview materials may further include a process of segment selection and duration fitting. The specific process includes: constructing a multidimensional scoring function and configuring adjustable weights for the multidimensional scoring function; using the multidimensional scoring function to quantify and score candidate segments and sorting them from high to low scores; using user intent data as constraints, employing a hybrid approach of greedy algorithm and dynamic programming to select a segment sequence from the sorted candidate segments; fine-tuning the segment sequence at the cut points and arranging the selected segments sequentially onto the video and audio tracks; generating subtitle tracks from the recognized text corresponding to the selected segments and writing labels into the timeline to generate a standard timeline file.

[0091] The computer device can construct a multi-dimensional scoring function that includes thematic relevance, information density, narrative coherence, audio-visual quality, character balance, emotional curve, semantic integrity, and compliance. Adjustable weights are assigned to each scoring dimension. Based on the scoring function, all candidate segments are quantitatively scored, resulting in a comprehensive score for each segment, which is then sorted from highest to lowest score. Next, the computer device, using a user-defined target duration as a constraint, and combining the cut-point positions of the candidate cut-point set with the smoothness scores corresponding to each cut-point, prioritizes candidate segments with high comprehensive scores to obtain the optimal segment sequence. Then, the computer device can fine-tune the cut-points of the optimal segment sequence, selecting cut-points with smoothness scores reaching a preset threshold as the final editing cut-points. The playback order of the segment sequence is then arranged according to narrative logic, matching optimal camera angles to multi-camera footage, adding standardized transition parameters, and simultaneously converting speech-recognized text into synchronized subtitles and binding them to corresponding timestamps. Finally, the computer equipment can generate standard timeline files in EDL / FCPXML / AAF / JSON format, including track layout, timeline markers, and metadata annotations, based on the fine-tuned sequence of clips, final cut points, camera parameters, transition parameters, and subtitle information.

[0092] In one embodiment, the intelligent editing method for long interview materials may further include a process of selecting a sequence of segments. The specific process includes: trimming the context buffer between time-out segments in the sorted candidate segments, supplementing candidate segments or merging adjacent semantically coherent segments for segments with insufficient duration, and obtaining a sequence of segments.

[0093] The computer equipment can perform segment combination and duration fitting iteration based on a hybrid strategy of greedy algorithm and dynamic programming. It can trim non-core context buffers and remove low-weight segments for overdue segments, and supplement the second-highest score candidate segments or merge adjacent semantically coherent segments for segments with insufficient duration. This ensures that the opening introduction and closing summary of the segment sequence are complete, and the final duration error is controlled within the preset tolerance range, thus obtaining the segment sequence.

[0094] In one embodiment, the intelligent editing method for long interview footage may also include the process of outputting a finished product. The specific process includes: decoding and re-encoding the audio and video footage according to a standard timeline file, outputting a low bitrate reference finished product, and outputting an embedded subtitle video to achieve a synchronized preview of the reference finished product and the subtitles.

[0095] Computer equipment can perform low-bitrate audio and video rendering based on standard timeline files, generating a previewable reference video and simultaneously outputting a subtitle preview file precisely aligned with the timeline of the final video, achieving synchronized previewing of the reference video and subtitles. Furthermore, it allows users to add review annotations in the preview interface, writing user preferences back into quantifiable parameter adjustments and triggering iterative optimization starting from the user's input.

[0096] In one embodiment, an intelligent editing method for long interview clips is provided, with an end-to-end technical process as follows: Figure 4 As shown, it mainly includes two parallel pipelines: a material preprocessing pipeline and a user intent-driven intelligent editing pipeline. The specific process includes:

[0097] Material Import: Receives user-uploaded interview-style audio and video / multi-camera raw materials;

[0098] Metadata registration: Records basic information such as timecode, frame rate, and camera position identifier of the footage to provide a benchmark for subsequent time synchronization;

[0099] Audio and video separation: Decapsulate the materials to separate independent audio and video streams, while strictly maintaining the time base of the two to avoid audio and video desynchronization;

[0100] Voice and speaker separation: Perform voice enhancement, speech activity detection (VAD), voiceprint extraction and speaker clustering on the audio stream, label host / guest roles, and mark overlapping speech and silent segments;

[0101] Automatic speech recognition and sentence segmentation: Perform multilingual ASR on the processed audio to generate punctuated text, word / sentence-level timestamps, and confidence scores;

[0102] Timestamp alignment and correction: Force alignment of word and phrase timestamps on the audio side with frame-level timestamps in the video stream, correct timestamp drift in long-running footage, generate candidate cut point sets and smoothness scores, and complete the entire preprocessing process;

[0103] User intent input (topic and duration): Receives core user input such as editing theme, target duration, style preferences, and compliance constraints;

[0104] Large-scale text understanding: Based on a large language model, it deeply analyzes user intent and extracts core demand keywords and editing direction;

[0105] Large-scale model summarization: Combining the preprocessed speech recognition text, key points, memorable quotes and narrative outlines are generated according to the interview narrative structure, and semantic deduplication and importance labeling are completed;

[0106] Text-time bidirectional mapping: It accurately binds the extracted summary key points with the preprocessed word and sentence timestamps, configures context buffers and alternative candidate segments for each mapping, and records the mapping confidence to achieve content traceability and replacement;

[0107] Timeline generation: Based on a multi-dimensional scoring function, candidate segments are scored, and combined with the candidate cut point set and smoothness score, the optimal segment sequence is selected within the target duration to generate a standardized editing timeline;

[0108] Output and pre-compositing: Generate standard timeline files such as EDL / FCPXML / AAF based on the timeline, render low bitrate reference footage and synchronized subtitle previews, and support user write-back preferences for iterative optimization.

[0109] This embodiment presents an intelligent, end-to-end automated method for editing long interview footage, integrating import and timeline export / pre-compositing to significantly reduce costs and increase efficiency. It optimizes for both topic and duration constraints, balancing information density and narrative coherence. Summary points are mapped one-to-one with sentence / word-level timestamps for easy review and refinement. It differentiates between host and guest, controlling speaking ratios and dialogue pace. Standard EDL / FCPXML / AAF export allows for seamless integration into existing NLEs. It includes audio-visual quality assessment, multilingual and sensitive content detection, and supports multi-camera setups, cloud acceleration, plug-in functionality, and API services. Effective implementation will significantly shorten the cycle of manual screening and repeated trial editing, improve the quality and consistency of the final cut, and provide a stable and reliable infrastructure for large-scale content production.

[0110] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0111] In one embodiment, such as Figure 5As shown, an intelligent editing system for long interview clips is provided, including: a clip acquisition and processing module 510, an audio processing module 520, an audio recognition and alignment module 530, a text and time mapping module 540, and a timeline generation and export module 550, wherein:

[0112] The material acquisition and processing module 510 is used to acquire imported audio and video materials, separate the audio stream and video stream, and maintain the consistency of the time base of the audio stream and video stream;

[0113] The audio processing module 520 is used to enhance human voice, detect and cluster speech activity in the audio stream, and obtain various voiceprint features in the audio stream. Based on the voiceprint features, the speaker is labeled with a role, and overlapping speech and silent segments are also labeled.

[0114] The audio recognition and alignment module 530 is used to perform multilingual automatic speech recognition on the audio stream, generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores, and force alignment and correction of word and sentence timestamps with timestamps in the video stream to generate a candidate cut point set and smoothness score.

[0115] The text and time mapping module 540 is used to collect user intent data, use a large language model to perform semantic understanding on the user intent data, extract text content and map it to word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence.

[0116] The timeline generation and export module 550 is used to score candidate segments based on a multi-dimensional scoring function. Based on the scoring results, it selects a segment sequence within the target duration based on the candidate cut point set and smoothness score, and generates a standard timeline file based on the segment sequence, outputting a reference finished film and subtitle preview.

[0117] In one embodiment, the material acquisition and processing module 510 is further configured to acquire imported audio and video materials and register material identifiers in the audio and video materials; decapsulate the audio and video materials, separate the audio stream and video stream, and maintain the temporal consistency of the audio stream and video stream based on the material identifiers.

[0118] In one embodiment, the audio processing module 520 is further configured to perform a short-time Fourier transform on the audio stream, generate a voice mask through a deep neural network, and recover the enhanced voice audio stream through an inverse transform to achieve voice enhancement; segment the enhanced voice audio stream into frames, extract acoustic features, and determine whether each frame is valid speech, dividing it into speech segments and non-speech segments; extract voiceprint features from each speech segment, perform speaker clustering based on the voiceprint features, and assign a unique speaker identifier to each cluster; map the speaker identifier to a specific role based on the voiceprint features, completing role labeling; perform overlap detection on the speech segments of multiple speakers, identify and mark overlapping speech segments; and mark non-speech segments as silent segments.

[0119] In one embodiment, the audio recognition and alignment module 530 is further configured to perform language recognition on the audio stream to obtain the language category, determine a multilingual automatic speech recognition model based on the language category for feature extraction, generate recognized text through a decoder; combine the pause and intonation information of the features extracted by the multilingual automatic speech recognition model to add punctuation marks to the recognized text; segment the text with punctuation marks into independent sentence units and record the start and end timestamps of each sentence unit; record the acoustic frame time position corresponding to each word and generate word-level start and end timestamps; and calculate the decoding matching probability of each word and sentence to generate word-sentence level confidence.

[0120] In one embodiment, the audio recognition and alignment module 530 is further configured to establish an initial mapping relationship between word / sentence timestamps and timestamps in the video stream based on a unified time base of the audio and video materials; divide the audio and video materials into various detection windows, calculate the drift deviation value between word / sentence timestamps and timestamps in the video stream, and perform forced correction on word / sentence timestamps based on the drift deviation value; collect sentence boundaries, word boundaries, silent segment boundaries, overlapping speech boundaries, and pause points as candidate cut-point sets based on the corrected word / sentence timestamps; and calculate the smoothness score of each candidate cut-point in the candidate cut-point set according to the dimensions of semantic integrity, speech continuity, energy smoothness, lip closure, and rhythm adaptability.

[0121] In one embodiment, the text-time mapping module 540 is further used to collect user input of topic descriptions, target durations, style preferences, role orientations, and compliance constraints to form structured user intent data; input the user intent data and multilingual automatic speech recognition results into a large language model, and complete the semantic understanding of user intent through the semantic parsing and similarity matching algorithms of the large language model, and extract text content based on semantic understanding; map the text content with the word and sentence timestamps in the multilingual automatic speech recognition results to establish a text-time mapping relationship table; for each mapping relationship in the mapping relationship table, set a preset market context buffer and retrieve alternative candidate segments in audio and video materials; generate the mapping confidence of each mapping relationship, and record the context buffer, alternative candidate segments, and mapping confidence in the mapping relationship table.

[0122] In one embodiment, the timeline generation and export module 550 is further configured to construct a multidimensional scoring function, configure adjustable weights for the multidimensional scoring function, use the multidimensional scoring function to quantify and score candidate segments and sort them from high to low scores; using user intent data as constraints, a segment sequence is selected from the sorted candidate segments using a hybrid approach of greedy algorithm and dynamic programming; the segment sequence is fine-tuned at the cut points, and the selected segments are arranged sequentially on the video track and audio track; the recognized text corresponding to the selected segments is used to generate a subtitle track, and tags are written into the timeline to generate a standard timeline file.

[0123] In one embodiment, the timeline generation and export module 550 is further used to trim the context buffer between time-out segments in the sorted candidate segments, supplement candidate segments for segments with insufficient duration, or merge adjacent semantically coherent segments to obtain a segment sequence.

[0124] In one embodiment, the timeline generation and export module 550 is also used to decode and re-encode audio and video materials according to a standard timeline file, output a low bitrate reference video, and output embedded subtitle video to achieve a linked preview of the reference video and subtitles.

[0125] In one embodiment, an intelligent editing system for long interview clips is provided, with the overall architecture as follows: Figure 6 As shown, a layered and modular architecture is adopted, from the top-level user interaction to the bottom-level support services, mainly including:

[0126] User interaction entry points: User / Administrator (Web / API), the system's external entry point, supporting access for ordinary users via the web interface and administrators via the API interface, used for submitting editing tasks, configuring parameters, viewing results, and iterative optimization.

[0127] Parameter configuration and interaction: It accepts user input and transforms requirements such as editing theme, target duration, style preference, and compliance constraints into structured parameters. It also supports users to write back the processing results and adjust their preferences.

[0128] Task orchestration / message bus: As the "central nervous system" of the system, it is responsible for task scheduling, distribution, status synchronization and message flow, orderly sending user parameters to the core processing cluster, and simultaneously sending back processing progress and results.

[0129] Processing Service Cluster: This is the core execution unit of the system, fully covering the entire process of intelligent pruning, and mainly includes:

[0130] (1) Material Acquisition and Management: At the starting point of the process, responsible for the import, verification and metadata registration of multi-camera / multi-format audio and video materials;

[0131] (2) Audio and video separation: Decapsulate the material, separate the audio stream and video stream, and maintain time base consistency.

[0132] (3) Sound source separation / speaker separation: Perform voice enhancement, VAD, voiceprint clustering, role labeling, and mark overlapping speech and silent segments;

[0133] (4) Speech recognition and alignment: Perform multilingual ASR, generate recognition results with punctuation, timestamps and confidence scores, and complete forced alignment and correction of audio and video timestamps;

[0134] (5) Topic Analysis and Summary: Analyze user intent through a large language model and extract core summaries, key phrases and narrative outlines;

[0135] (6) Text-time mapping: Bi-directionally bind summary key points with word and sentence timestamps, configure context buffers and alternative candidate segments, and record mapping confidence;

[0136] (7) Segment selection and rhythm control: Based on the multi-dimensional scoring function, the candidate segments are scored, and the optimal segment sequence is selected within the target duration;

[0137] (8) Quality assessment and security: Perform audio and video quality testing, identify slips of the tongue / redundant words, and conduct compliance testing of sensitive content to ensure that the final product meets the release requirements;

[0138] (9) Timeline generation and export: Generate standard timeline files such as EDL / FCPXML / AAF / JSON, and complete the configuration of tracks, transitions and subtitles;

[0139] (10) Pre-compositing and playback: Render low bitrate reference films and subtitle previews, supporting user preview and iterative optimization.

[0140] Model services: These services support core AI capabilities, including pre-trained models such as sound source separation, ASR (Automatic Speech Recognition), LLM (Large Language Model), and compliance detection, providing algorithmic support for upper-layer business modules.

[0141] Data and Storage: Responsible for the storage and management of data throughout the entire process, including metadata database, object storage, caching, and log auditing, ensuring the security and traceability of materials, intermediate results, and configuration parameters.

[0142] External interoperability: Achieves seamless integration with mainstream industry tools, supporting direct import and refinement of non-linear editing systems (Premiere / FCP / Resolve), audio workstations (Pro Tools), and subtitle and review systems to meet the needs of industrial production.

[0143] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an intelligent editing method for long interview-style materials. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0144] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0145] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of an intelligent editing method for long interview materials.

[0146] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to implement the steps of an intelligent editing method for long interview materials.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0149] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for intelligent editing of long interview-style footage, characterized in that, The method includes: The imported audio and video materials are obtained, the audio stream and video stream are separated, and the temporal base of the audio stream and video stream are kept consistent. The audio stream is enhanced with human voice, and speech activity is detected and clustered. Each voiceprint feature in the audio stream is obtained. The speaker is labeled with the voiceprint features, and overlapping speech and silent segments are also labeled. The audio stream is subjected to multilingual automatic speech recognition to generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores. The word and sentence timestamps are then forcibly aligned and corrected with the timestamps in the video stream to generate a candidate cut point set and a smoothness score. Collect user intent data, use a large language model to perform semantic understanding on the user intent data, extract text content and map it to the word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence. The candidate segments are scored using a multidimensional scoring function. Based on the scoring results, a segment sequence is selected within the target duration based on the candidate cut point set and smoothness score. A standard timeline file is generated based on the segment sequence, and a reference finished product and subtitle preview are output.

2. The intelligent editing method for long interview materials according to claim 1, characterized in that, The process involves acquiring imported audio and video materials, separating the audio and video streams, and maintaining temporal consistency between the audio and video streams, including: Obtain the imported audio and video materials, and register the material identifiers in the audio and video materials; The audio and video materials are decapsulated to separate the audio stream and video stream, and the temporal base consistency of the audio stream and video stream is maintained based on the material identifier.

3. The intelligent editing method for long interview materials according to claim 1, characterized in that, The audio stream is subjected to voice enhancement, speech activity detection, and clustering. Various voiceprint features are obtained from the audio stream, and speaker roles are assigned based on these features. Overlapping speech segments and silent segments are also labeled, including: The audio stream is subjected to a short-time Fourier transform, a voice mask is generated by a deep neural network, and the enhanced voice audio stream is recovered by inverse transform to achieve voice enhancement. The audio stream after voice enhancement is framed, acoustic features are extracted, and it is determined whether each frame is valid speech, and speech segments and non-speech segments are divided. Voiceprint features are extracted from each of the speech segments, and speaker clusters are performed based on the voiceprint features. A unique speaker identifier is assigned to each cluster. Based on the voiceprint features, the speaker identifier is mapped to a specific role, thus completing the role labeling; Overlap detection is performed on the speech segments of multiple speakers, overlapping speech segments are identified and marked; the non-speech segments are marked as silent segments.

4. The intelligent editing method for long interview materials according to claim 1, characterized in that, Perform multilingual automatic speech recognition on the audio stream to generate recognition results including punctuation marks, word and sentence timestamps, and confidence levels, including: The audio stream is subjected to language recognition to obtain the language category. Based on the language category, a multilingual automatic speech recognition model is determined to extract features, and the recognized text is generated by the decoder. The pause and intonation information extracted from the multilingual automatic speech recognition model are combined to add punctuation marks to the recognized text; The text containing the punctuation marks is segmented into independent sentence units, and the start and end timestamps of each sentence unit are recorded. Record the acoustic frame time position corresponding to each word, and generate word-level start and end timestamps; Calculate the decoding matching probability of each word and phrase, and generate word-phrase level confidence scores.

5. The intelligent editing method for long interview materials according to claim 1, characterized in that, The timestamps of the stated words and phrases are forcibly aligned and corrected with the timestamps in the video stream to generate a candidate cut-off point set and a smoothness score, including: Based on the unified time base of the audio and video materials, an initial mapping relationship is established between the word timestamps and the timestamps in the video stream; The audio and video materials are divided into various detection windows. The drift deviation value between the word and phrase timestamps and the timestamps in the video stream is calculated, and the word and phrase timestamps are forcibly corrected based on the drift deviation value. Based on the corrected word and sentence timestamps, sentence boundaries, word boundaries, silent segment boundaries, overlapping speech boundaries, and pause points are collected as candidate cut point sets; Based on the dimensions of semantic integrity, speech continuity, energy smoothness, mouth closure degree, and rhythmic adaptability, the smoothness score of each candidate cut point in the candidate cut point set is calculated.

6. The intelligent fine-editing method for long interview materials according to claim 1, characterized in that, User intent data is collected, and semantic understanding of the user intent data is performed using a large language model. Text content is extracted and mapped to word and sentence timestamps. Contextual buffers and alternative candidate segments are added to each mapping, and mapping confidence is recorded, including: Collect user input including topic descriptions, target duration, style preferences, role orientations, and compliance constraints to form structured user intent data; The user intent data and multilingual automatic speech recognition results are input into a large language model. The semantic parsing and similarity matching algorithm of the large language model are used to complete the semantic understanding of the user intent and extract the text content based on the semantic understanding. The text content is mapped to the word and phrase timestamps in the multilingual automatic speech recognition results to establish a text-time mapping table; For each mapping relationship in the mapping table, a context buffer space of a preset market is set, and alternative candidate segments are retrieved from the audio and video materials; Generate the mapping confidence score for each of the mapping relationships, and record the context buffer, alternative candidate segments, and mapping confidence scores into the mapping relationship table.

7. The intelligent editing method for long interview materials according to claim 1, characterized in that, The candidate segments are scored using a multidimensional scoring function. Based on the scoring results, and using the candidate cut-off point set and smoothness score, a segment sequence is selected within the target duration. A standard timeline file is then generated based on the segment sequence, including: Construct a multidimensional scoring function and configure adjustable weights for the multidimensional scoring function. Use the multidimensional scoring function to quantify and score the candidate segments and sort them from high to low scores. Using the user intent data as a constraint, a sequence of segments is selected from the sorted candidate segments using a combination of greedy algorithm and dynamic programming. The segment sequence is fine-tuned at the cut points, and the selected segments are arranged sequentially on the video track and audio track. The recognized text corresponding to the selected segments is used to generate a subtitle track, and tags are written into the timeline to generate a standard timeline file.

8. The intelligent editing method for long interview materials according to claim 7, characterized in that, Select a sequence of segments from the sorted candidate segments, including: The context buffer is trimmed from the timed-out segments in the sorted candidate segments, and candidate segments are added to segments with insufficient duration or adjacent semantically coherent segments are merged to obtain the segment sequence.

9. The intelligent editing method for long interview materials according to claim 7, characterized in that, The method further includes: Based on the standard timeline file, the audio and video materials are decoded and re-encoded to output a low-bitrate reference video and an embedded subtitle video, enabling a synchronized preview of the reference video and the subtitles.

10. An intelligent editing system for long interview clips, characterized in that, The system includes: The material acquisition and processing module is used to acquire imported audio and video materials, separate the audio stream and video stream, and maintain the consistency of the time base of the audio stream and video stream; The audio processing module is used to enhance human voice, detect and cluster speech activity in the audio stream, and obtain various voiceprint features in the audio stream. Based on the voiceprint features, the speaker is labeled with a role, and overlapping speech and silent segments are also labeled. The audio recognition and alignment module is used to perform multilingual automatic speech recognition on the audio stream, generate recognition results containing punctuation marks, word and sentence timestamps, and confidence scores, and forcibly align and correct the word and sentence timestamps with the timestamps in the video stream to generate a candidate cut point set and a smoothness score. The text and time mapping module is used to collect user intent data, perform semantic understanding on the user intent data using a large language model, extract text content and map it to the word and sentence timestamps, add context buffers and alternative candidate segments to each mapping, and record the mapping confidence. The timeline generation and export module is used to score the candidate segments based on a multi-dimensional scoring function. Based on the scoring results, the module selects a segment sequence within the target duration based on the candidate cut point set and smoothness score, generates a standard timeline file based on the segment sequence, and outputs a reference finished product and subtitle preview.

Images