An AI vision model and scene detection fusion-based video opening and closing intelligent identification method and system

Abstract

This invention discloses a method and system for intelligent recognition of video title and end credits based on the fusion of AI visual models and scene detection. The method includes: Step 1: Adaptively adjusting the candidate detection regions for the title and end credits using an adaptive candidate region adjustment algorithm to obtain a title and end credits candidate region; Step 2: Extracting title video segments from the title candidate region at a set sampling interval, and extracting end video segments from the end candidate region at a set sampling interval; Step 3: Using an AI visual language model to perform content understanding on the sampled title video segments, identifying the title visual feature type and feature level of the segments. This invention establishes a multi-level judgment mechanism based on feature weights, fully utilizing the reliability differences of different visual features to achieve accurate recognition of title and end credits time points, significantly improving the accuracy of title and end credits recognition.

Description

Technical Field

[0001] This invention belongs to the field of video content processing and relates to a method and system for intelligent recognition of video title and ending sequences based on the fusion of AI visual models and scene detection. Background Technology

[0002] With the rapid development of video platforms and OTT services, users' demands for viewing experience are constantly increasing. In video playback scenarios, features such as automatically skipping intros and outros and enabling continuous playback of episodes have become important means to improve user experience. Meanwhile, in media asset management and content distribution scenarios, accurately identifying the timing of intros and outros is of significant value for content analysis, copyright protection, and advertising.

[0003] With diverse video content formats and significant differences in the length of intros and outros, traditional fixed-length trimming methods cannot adapt to this diversity and have low accuracy. While manual annotation is accurate, it is costly and cannot meet the real-time requirements of news editing and program scheduling. Feature-based detection methods work well for fixed-pattern intros and outros (such as using the same intro animation for each episode), but are poorly adaptable to video content with varied intro and outros formats. Single judgment methods based on AI visual models may misjudge intro and outros features and suffer from problems such as failing to fully utilize the reliability differences of different features, failing to ensure that the trimming point is at the scene boundary, and low processing efficiency. Especially in scenarios such as multi-channel parallel broadcasting and rapid emergency news production, higher requirements are placed on the accuracy and real-time performance of intelligent intro and outros recognition. Therefore, there is an urgent need for an intelligent detection solution that can accurately identify the time points of intros and outros, ensure the integrity of the trimmed footage, and take into account processing efficiency. Summary of the Invention

[0004] In order to overcome at least one deficiency of the prior art, the present invention provides a method and system for intelligent recognition of video title and end credits based on the fusion of AI visual models and scene detection.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for intelligent recognition of video intros and outros based on the fusion of AI visual models and scene detection includes the following steps: Step 1: Adaptively adjust the candidate detection regions of the beginning and end of the film based on the adaptive candidate region adjustment algorithm to obtain the candidate regions of the beginning and end of the film; Step 2: Extract the opening video segment from the opening candidate area at the set sampling interval, and extract the closing video segment from the closing candidate area at the set sampling interval; Step 3: Use an AI visual language model to perform content understanding on the sampled intro video clips, and identify the type and level of visual features of the intro clips; Step 4: Establish a title sequence detection algorithm with feature weights, perform title sequence detection based on feature levels, and coarsely locate the end time point of the title sequence; Step 5: Use an AI visual language model to analyze the sampled end-credits video clips and identify their features; Step 6: Establish an asymmetric threshold algorithm for opening sequence detection, and perform opening sequence detection based on the segment features of the closing sequence to roughly locate the starting time point of the closing sequence; Step 7: Establish a scene boundary optimization algorithm to determine the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning beginning clip end time point and the coarse positioning end clip start time point.

[0006] Furthermore, the method for adaptively adjusting the candidate detection regions of the beginning and end of the film based on the adaptive candidate region adjustment algorithm in step 1 is as follows: Step 11: Determine whether the total duration of the candidate area is not less than the total duration of the video. If yes, adopt the proportional strategy and proceed to step 12. If no, adopt the fixed duration strategy and proceed to step 13. Step 12: Set the opening sequence candidate area = total video duration × 1 / 3, and the closing sequence candidate area = total video duration × 1 / 3; Step 13: Set the opening sequence candidate area to a fixed duration A and the closing sequence candidate area to a fixed duration B.

[0007] Furthermore, the method for coarsely locating the end time point of the opening sequence using the feature weight opening sequence detection algorithm includes the following steps: Step 41: Construct a weighting system for visual features, and assign different reliability weights to features based on their level to obtain high-reliability features, medium-reliability features, and low-reliability features. Step 42: Establish a dynamic termination mechanism for high reliability features to perform in-chip detection; Step 43: Establish a continuous verification mechanism for medium and low reliability characteristics to perform header detection.

[0008] Furthermore, the dynamic termination mechanism immediately terminates the subsequent chip header scan when a high reliability feature is detected, and records the time point where the feature is located as the end time point of the coarse positioning chip header.

[0009] Furthermore, the continuous confirmation mechanism sets a continuous threshold for medium reliability features and a continuous threshold for low reliability features. The continuous threshold for medium reliability features is greater than the continuous threshold for low reliability features. When the number of continuous confirmations for medium reliability features or low reliability features reaches the continuous threshold for medium reliability features or the continuous threshold for low reliability features, it is determined as the end point of the beginning of the segment, and the coarse positioning end point of the beginning of the segment is recorded.

[0010] Furthermore, the asymmetric threshold start-of-episode detection algorithm in step 6, which performs end-of-episode detection based on the segment features of the end-of-episode and coarsely locates the start time point of the end-of-episode, includes the following steps: Step 61: Set up a forward counter and a reverse counter to detect and count the end-of-film features and the main-film features, respectively; Step 62: Establish reset rules: The positive counter increments by 1 when it detects the end-of-film feature and resets to 0 when it detects the main film feature; the negative counter increments by 1 when it detects the main film feature and resets to 0 when it detects the end-of-film feature. Step 63: Set the confirmation threshold for the middle of the end-of-film candidate area, the confirmation threshold for the end of the end-of-film candidate area, and the confirmation threshold for the main film. The confirmation threshold for the middle of the end-of-film candidate area is greater than the confirmation threshold for the end of the end-of-film candidate area. Step 64: Determine whether the segment feature is located in the middle of the end candidate area. If yes, proceed to step 65; otherwise, proceed to step 66. Step 65: When the positive counter continuously reaches the confirmation threshold in the middle of the end-of-film candidate area, it is determined that the end of the film has been reached. The first time point reached by the positive counter is used to coarsely locate the start time point of the end of the film. Step 66: When the forward counter continuously reaches the end confirmation threshold of the end candidate area, it is determined that the end of the film has been reached. The first time point reached by the forward counter is used to coarsely locate the start time point of the end of the film.

[0011] Furthermore, step 7, establishing a scene boundary optimization algorithm, and determining the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning beginning clip end time point and the coarse positioning end clip start time point, includes the following steps: Step 71: Set time tolerances before and after the end time of the coarse positioning beginning and the start time of the coarse positioning end; Step 72: Identify the differences between adjacent frames within the time tolerance, and determine whether the differences reach the set threshold. If yes, determine that there is a scene switching boundary and proceed to step 73; otherwise, determine that there is no scene switching boundary and proceed to step 74. Step 73: Calculate the time difference between all scene transition boundaries and the coarse positioning start time point or coarse positioning end time point, and extract the boundary with the smallest time difference as the start time point or end time point. Step 74: Use the boundary where the coarse positioning beginning time point ends or the coarse positioning ending time point begins as the beginning end cropping boundary or the ending start cropping boundary.

[0012] A video title and end credit intelligent recognition system based on the fusion of AI visual models and scene detection, including Candidate region localization module: Based on the total video duration, adaptively adjust the candidate detection regions at the beginning and end of the video to obtain the candidate regions at the beginning and end of the video. Video sampling module: Extracts the opening video segment from the opening candidate area according to the sampling interval, and extracts the closing video segment from the closing candidate area according to the sampling interval; AI Semantic Analysis Module: Uses AI visual language models to understand the content of sampled intro video clips and identify the types of visual features in the intro of the clips; Feature weight judgment module: Based on the feature types identified by the AI ​​semantic analysis module, different reliability weights are assigned, and a differentiated confirmation strategy is executed based on the differentiated weight features to roughly locate the end time point of the opening sequence. Dynamic Termination Module: Real-time monitoring feature weight judgment module. When a high reliability feature is detected, the subsequent header scan is immediately terminated, and the time point where the feature is located is recorded as the header end time point of coarse positioning. Asymmetric threshold strategy module: Uses AI visual language model to analyze the sampled end-credits video clips to identify clip features, tracks and detects clip features based on a bidirectional counter mechanism, and coarsely locates the start time point of the end-credits video. Scene boundary optimization module: For the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro, the scene detection algorithm is used to locate the scene switching boundary within the time tolerance range, or the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro are used as the scene switching boundary.

[0013] Furthermore, the AI ​​semantic analysis module identifies multi-level visual feature types for the opening sequence, including strong semantic markers, fixed visual patterns, and weak stylistic features; feature levels are set according to the visual feature types, including first-level features, second-level features, and third-level features.

[0014] Furthermore, the feature weight judgment module assigns the first-level feature as a high-reliability feature, the second-level feature as a medium-reliability feature, and the third-level feature as a low-reliability feature. The high-reliability feature uses a single confirmation strategy for header detection, while the medium-reliability feature and the low-reliability feature use a continuous confirmation strategy for header detection.

[0015] In summary, the advantages of this invention are: This invention establishes a multi-level judgment mechanism based on feature weights, making full use of the reliability differences of different visual features to achieve accurate identification of the opening and closing time points, significantly improving the accuracy of opening and closing time point identification.

[0016] This invention solves the problem of incomplete images caused by the cropping point being located in the middle of the scene transition in traditional methods through a two-layer optimization that combines AI semantic analysis and scene boundary optimization. It ensures the semantic accuracy of the time point and the visual coherence of the cropped video. At the same time, the coarse positioning time point is used as a fault-tolerant design to fall back to the coarse positioning time point when there is no scene boundary within the search window, thus ensuring system availability.

[0017] This invention significantly reduces computational costs and greatly improves processing efficiency while ensuring accuracy, through the dynamic early termination mechanism of the dynamic termination module and the asymmetric threshold strategy of the asymmetric threshold strategy module.

[0018] This invention has good adaptability, scalability, automation capabilities, and low operating costs. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the workflow of the intelligent video intro and outro recognition method of the present invention.

[0020] Figure 2 This is a schematic diagram of the workflow of the scene boundary optimization algorithm of the present invention.

[0021] Figure 3 This is a schematic diagram of the intelligent video intro and outro recognition system module architecture of the present invention. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0023] Example: like Figures 1-2 As shown, a method for intelligent recognition of video title and ending sequences based on the fusion of AI visual models and scene detection includes the following steps: Step 1: Adaptively adjust the candidate detection regions of the beginning and end of the film based on the adaptive candidate region adjustment algorithm to obtain the candidate regions of the beginning and end of the film; Step 2: Extract the opening video segment from the opening candidate area at a set sampling interval (e.g., every few seconds), and extract the closing video segment from the closing candidate area at a set sampling interval (e.g., every few seconds). Step 3: Use an AI visual language model to perform content understanding on the sampled intro video clips, and identify the type and level of visual features of the intro clips; The types of visual features in the opening sequence include strong semantic markers such as the text "Episode X" and the title text, as well as fixed visual patterns such as the opening animation style and fixed sequence of opening images, and weak stylistic features such as color tone, visual style, background music and subtitles. Feature levels are set according to the type of visual feature, including first-level features, second-level features and third-level features.

[0024] Step 4: Establish a title sequence detection algorithm with feature weights, perform title sequence detection based on feature levels, and coarsely locate the end time point of the title sequence; Step 5: Use an AI visual language model to analyze the sampled end-credits video clips and identify their features; Step 6: Establish an asymmetric threshold algorithm for opening sequence detection, and perform opening sequence detection based on the segment features of the closing sequence to roughly locate the starting time point of the closing sequence; Step 7: Establish a scene boundary optimization algorithm to determine the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning beginning clip end time point and the coarse positioning end clip start time point.

[0025] The steps of the method for adaptively adjusting the candidate detection regions of the beginning and end of the video based on the adaptive candidate region adjustment algorithm are as follows: Step 11: Determine whether the total duration of the candidate area is not less than the total duration of the video. If yes, adopt the proportional strategy and proceed to step 12. If no, adopt the fixed duration strategy and proceed to step 13. Total candidate region duration = Pre-roll candidate region duration + Post-roll candidate region duration; Step 12: Set the opening sequence candidate area = total video duration × 1 / 3, and the closing sequence candidate area = total video duration × 1 / 3; Step 13: Set the opening sequence candidate area to a fixed duration A and the closing sequence candidate area to a fixed duration B.

[0026] The steps of the feature-weighted in-game title sequence detection algorithm for coarsely locating the end time point of the title sequence include: Step 41: Construct a weighting system for visual features, and assign different reliability weights to features based on their level to obtain high-reliability features, medium-reliability features, and low-reliability features. Step 42: Establish a dynamic termination mechanism for high reliability features to perform in-chip detection; Dynamic termination mechanism: When a high reliability feature is detected, the subsequent chip header scan is immediately terminated, and the time point where the feature is located is recorded as the end time point of the coarse positioning chip header; Step 43: Establish a continuous verification mechanism for medium and low reliability characteristics to perform header detection.

[0027] Continuous confirmation mechanism: Set continuous thresholds for medium reliability features and low reliability features. The continuous threshold for medium reliability features is greater than the continuous threshold for low reliability features. When the number of continuous confirmations for medium reliability features or low reliability features reaches the continuous threshold for medium reliability features or low reliability features, it is determined as the end point of the title sequence, and the coarse positioning title sequence end time point is recorded.

[0028] The asymmetric threshold algorithm for detecting the beginning of a segment, based on the segment features of the ending, performs ending detection and coarsely locates the start time point of the ending segment. The method includes the following steps: Step 61: Set up a forward counter and a reverse counter to detect and count the end-of-film features and the main-film features, respectively; Step 62: Establish reset rules: The positive counter increments by 1 when it detects the end-of-film feature and resets to 0 when it detects the main film feature; the negative counter increments by 1 when it detects the main film feature and resets to 0 when it detects the end-of-film feature. Step 63: Set the confirmation threshold for the middle of the end-of-film candidate area, the confirmation threshold for the end of the end-of-film candidate area, and the confirmation threshold for the main film. The confirmation threshold for the middle of the end-of-film candidate area is greater than the confirmation threshold for the end of the end-of-film candidate area. Step 64: Determine whether the segment feature is located in the middle of the end candidate area. If yes, proceed to step 65; otherwise, proceed to step 66. Step 65: When the positive counter continuously reaches the confirmation threshold in the middle of the end-of-film candidate area, it is determined that the end of the film has been reached. The first time point reached by the positive counter is used to coarsely locate the start time point of the end of the film. Step 66: When the forward counter continuously reaches the end confirmation threshold of the end candidate area, it is determined that the end of the film has been reached. The first time point reached by the forward counter is used to coarsely locate the start time point of the end of the film.

[0029] The method for establishing a scene boundary optimization algorithm, which determines the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning end time point of the beginning clip and the coarse positioning start time point of the end clip, includes the following steps: Step 71: Set time tolerances before and after the end time of the coarse positioning beginning and the start time of the coarse positioning end; Step 72: Identify the differences between adjacent frames within the time tolerance, and determine whether the differences reach the set threshold. If yes, determine that there is a scene switching boundary and proceed to step 73; otherwise, determine that there is no scene switching boundary and proceed to step 74. Step 73: Calculate the time difference between all scene transition boundaries and the coarse positioning start time point or coarse positioning end time point, and extract the boundary with the smallest time difference as the start time point or end time point. Step 74: Use the boundary where the coarse positioning beginning time point ends or the coarse positioning ending time point begins as the beginning end cropping boundary or the ending start cropping boundary.

[0030] This application automates the entire process of annotation, replaces manual annotation, uses a unified judgment standard to ensure annotation consistency, supports real-time processing and large-scale deployment, and significantly reduces labor costs and improves processing timeliness for large-scale video platforms.

[0031] This application provides precise timing points for automatic skip and continuous playback functions. Through feature weighting system and scene boundary optimization, it ensures that the main content is not skipped and the picture is complete. It supports personalized function configuration, improves user satisfaction and platform competitiveness.

[0032] like Figure 3 As shown, this application also provides a video title and end credit intelligent recognition system based on the fusion of AI visual models and scene detection, including... Candidate region localization module: Based on the total video duration, adaptively adjust the candidate detection regions at the beginning and end of the video to obtain the candidate regions at the beginning and end of the video. Video sampling module: Extracts the opening video segment from the opening candidate area at sampling intervals (e.g., every few seconds), and extracts the closing video segment from the closing candidate area at sampling intervals (e.g., every few seconds). AI Semantic Analysis Module: Uses AI visual language models to understand the content of sampled intro video clips and identify the types of visual features in the intro of the clips; Feature weight judgment module: Based on the feature types identified by the AI ​​semantic analysis module, different reliability weights are assigned, and a differentiated confirmation strategy is executed based on the differentiated weight features to roughly locate the end time point of the opening sequence. Dynamic Termination Module: Real-time monitoring feature weight judgment module. When a high reliability feature is detected, the subsequent header scan is immediately terminated, and the time point where the feature is located is recorded as the header end time point of coarse positioning. Asymmetric threshold strategy module: Uses AI visual language model to analyze the sampled end-credits video clips to identify clip features, tracks and detects clip features based on a bidirectional counter mechanism, and coarsely locates the start time point of the end-credits video. Scene boundary optimization module: For the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro, the scene detection algorithm is used to locate the scene switching boundary within the time tolerance range, or the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro are used as the scene switching boundary.

[0033] The intro candidate region is the specified duration starting from the beginning of the video, and the outro candidate region is the specified duration before the end of the video. Based on the defined total video length, videos are divided into long and short videos. The candidate region localization module adjusts the candidate detection regions for the intro and outro using an adaptive adjustment strategy. This strategy employs a fixed duration for long videos and a proportional strategy (each region occupies a certain proportion of the total video duration) for short videos, ensuring the system's adaptability to videos of different lengths. The adaptive adjustment strategy provides scenario-based parameter configurations for different video types, and the feature keywords can be dynamically expanded, ensuring broad applicability to various video content.

[0034] This application only uses the candidate areas at the beginning and end of the video as the detection areas, and does not perform a full scan of the entire video, which greatly reduces the number of video segments that need to be processed.

[0035] The sampling interval of the video sampling module can be dynamically adjusted according to the video type. Dense sampling is used for TV series and sparse sampling is used for movies.

[0036] The sampling interval is configurable. For time-sensitive business scenarios, the sampling interval can be increased appropriately (e.g., from 5 seconds to 10 seconds), sacrificing a small amount of accuracy for faster processing speed; for scenarios with high accuracy requirements, the sampling interval can be reduced.

[0037] The AI ​​semantic analysis module can identify multi-level visual feature types in the opening sequence. These visual feature types include strong semantic markers such as the text "Episode X" and the title text, as well as fixed visual patterns such as the opening animation style and fixed sequence of opening images, and weak stylistic features such as color tone, visual style, background music, and subtitles. Feature levels are set according to the visual feature type, including first-level features, second-level features, and third-level features.

[0038] The feature weight judgment module assigns different reliability weights as follows: first-level features are assigned high reliability, second-level features are assigned medium reliability, and third-level features are assigned low reliability.

[0039] The differentiated confirmation strategy specifically employs a single confirmation strategy for highly reliable primary features, where the detection of a primary feature is sufficient to determine the end point of the title sequence. For secondary and tertiary features, a continuous confirmation strategy is adopted, setting different thresholds for consecutive title sequences. The threshold for secondary features is greater than that for tertiary features. When the analysis results of multiple consecutive time points are consistent, the title sequence end point is determined, and the coarse location of the title sequence end point is recorded.

[0040] The threshold for continuous sequence of secondary and tertiary features can be flexibly adjusted according to video type and business scenario.

[0041] Table 1 is a list of corresponding feature types, feature levels, reliability weights, and differentiated confirmation strategies; Table 1

[0042] This application employs a single-confirmation strategy for its highly reliable primary features. The dynamic termination module can determine the end point of the chip header and terminate subsequent scans upon detecting the primary feature, eliminating the need for continuous verification. These features exhibit extremely high reliability and a very low false positive rate, enabling the determination of the chip header end position immediately upon detection, significantly reducing computational costs.

[0043] The dynamic termination module saves computational resources by terminating early: Traditional methods require a full scan of the entire candidate region (e.g., the first 600 seconds of the opening sequence), and even after the end position of the opening sequence is determined, subsequent segments still need to be analyzed. The dynamic termination module of this invention terminates the scan immediately upon detecting high-weight features (e.g., the text "Episode X"), avoiding unnecessary AI model calls. Efficiency improvement example: Assuming the opening sequence ends at 120 seconds, the system detects the text "Episode X" at that position and immediately terminates the scan. Compared to a full scan of the 600-second candidate region, this saves 480 seconds of sampling and analysis, significantly reducing computational costs.

[0044] This application employs a continuous confirmation strategy for medium-reliability and low-reliability features, requiring consistent analysis results from multiple consecutive sampling points before a decision is made. This strategy effectively filters out accidental misjudgments by the AI ​​model, significantly reducing the false positive rate.

[0045] This application can simultaneously identify and utilize visual features at three levels: primary, secondary, and tertiary. Even if a video lacks high-weight features (such as some variety shows that do not have clear episode number identifiers), it can still complete the detection through continuous confirmation of medium and low-weight features, demonstrating strong adaptability.

[0046] Compared to fixed-length cropping methods, this invention makes judgments based on the actual content characteristics of the video, which can adapt to the differences in the length of the opening and closing credits of different types of videos (TV series, variety shows, movies, documentaries, etc.), and can also handle the changes in the length of the opening credits of different episodes of the same series, making it adaptable to the diversity of video content.

[0047] In practical applications, this invention significantly improves recognition accuracy compared to traditional fixed-duration cropping methods; and compared to single AI model judgment methods, it effectively reduces the false judgment rate through feature weights and continuous confirmation strategies.

[0048] In practical applications, the processing time for videos that detect high-weight features (which account for the majority) is significantly shorter than that of the full-scan method; even for videos lacking high-weight features, the processing efficiency is better than traditional AI full-video analysis methods through asymmetric thresholding strategy and candidate region limitation.

[0049] Asymmetric threshold strategy module: Uses AI visual language model to analyze the sampled end-credits video clips to identify clip features, tracks and detects clip features based on a bidirectional counter mechanism, and coarsely locates the start time point of the end-credits video. Because the content characteristics of the opening and closing credits differ significantly, an asymmetric and differentiated consecutive confirmation threshold is designed. The opening credits typically have continuous and fixed content, so a higher consecutive confirmation threshold is used; the closing credits may contain content from the main content, so a lower consecutive confirmation threshold is used, and the threshold requirement for the closing credits video segments is reduced to improve recall while ensuring accuracy.

[0050] Table 2 lists the differences in content features between the opening and closing credits; Table 2

[0051] The AI ​​visual language model analyzes and identifies segment features, including end-of-episode features and main-episode features. A two-way counter mechanism detects end-of-episode features and main-episode features and records the number of times. End-of-episode features are determined according to the set reset rules.

[0052] End credits features include cast and crew credits, special filters, lyrics display, fade-in / fade-out effects, and sudden changes in visual style; main feature features include character dialogues, plot content, and other iconic elements. AI visual language models are used to analyze video clips to identify end credits features and main feature features.

[0053] The specific reset rules are as follows: Set a continuous end-of-film threshold and a positive-film confirmation threshold. Use a positive counter and a negative counter to detect end-of-film features and positive-film features respectively. The positive counter increments by 1 when it detects an end-of-film feature and resets to 0 when it detects a positive-film feature. The negative counter increments by 1 when it detects a positive-film feature and resets to 0 when it detects an end-of-film feature. When the positive counter continuously reaches the continuous end-of-film threshold, it is determined to be the start point of the end-of-film. When the negative counter reaches the positive-film confirmation threshold, it indicates that the current position is still within the positive film, and scanning continues. Table 3 shows an example of the state transitions for tracking and detecting segment features using the bidirectional counter mechanism.

[0054] Table 3

[0055] Taking a continuous threshold of 3 times for the end of the film and a confirmation threshold of 2 times for the main film as an example, Table 3 shows that when the main film feature is detected at time 4, the positive counter is reset to 0 and the negative counter is incremented by 1, indicating that the main film interference occurred during the detection process. The scanning continues, and at time 7, the positive counter continuously reaches the continuous threshold for the end of the film, indicating that the end of the film has been reached. The first time point when the positive counter continuously reaches the continuous threshold for the end of the film is taken as the start point of the end of the film, i.e., time 5. As can be seen from the above, even if the main film interference occurs during the detection process (time 4), the system can eliminate false judgments through the reset rule of the bidirectional counter mechanism and finally coarsely locate the start time point of the end of the film. This application improves robustness by simultaneously tracking the end of the film and the main film for confirmation.

[0056] This application uses an asymmetric design for the end-of-episode and beginning-of-episode continuous thresholds, setting the end-of-episode continuous threshold to be lower than the beginning-of-episode continuous threshold to accommodate the differences in content features between the beginning and end of the episode. The higher beginning-of-episode continuous threshold ensures the strictness of the beginning-of-episode detection, while the lower end-of-episode continuous threshold adapts to the diversity of end-of-episode features, allowing for determination with fewer consecutive confirmations of the end-of-episode, thus accelerating the end-of-episode detection speed and improving the recall rate.

[0057] Table 4 illustrates the asymmetric threshold design based on the differences in content features between the opening and closing credits. Table 4

[0058] This application sets differentiated end-of-screen continuous thresholds based on the position of segment features in the end-of-screen candidate region. The end-of-screen continuous thresholds are divided into a confirmation threshold for the middle of the end-of-screen candidate region and a confirmation threshold for the end of the end-of-screen candidate region. The confirmation threshold for the middle of the end-of-screen candidate region is greater than the confirmation threshold for the end of the end-of-screen candidate region. Therefore, when a segment feature moves from the middle of the end-of-screen candidate region to the end of the end-of-screen candidate region, it automatically switches from the higher confirmation threshold for the middle of the end-of-screen candidate region to the lower confirmation threshold for the end of the end-of-screen candidate region, thereby enabling faster identification of the end-of-screen start point near the end of the video. This application also automatically lowers the threshold at the end of the end-of-screen candidate region to achieve dynamic threshold adjustment, effectively improving the recall rate.

[0059] The last two sampled segments of the end-of-film candidate region are defined as the tail of the end-of-film candidate region, and the rest are defined as the middle of the end-of-film candidate region.

[0060] The middle and end of the end credits candidate area are set according to the actual situation.

[0061] The AI ​​visual language model samples at sampling intervals, which may cause the coarse localization time point to fall in the middle of a scene transition. Direct cropping will result in transitional scenes such as black screens and fade-in / fade-out. The scene boundary optimization module sets a time tolerance before and after the coarse localization time point (e.g., a few seconds before and after, 10-15 seconds each), and the time tolerance is the size of the search window. Within this search window, the scene detection algorithm is used to identify significant differences between adjacent frames and locate the scene transition boundary. If a scene transition boundary exists within the search window, the boundary closest to the coarse localization time point is selected as the final cropping boundary, achieving a dual guarantee of semantic accuracy and visual coherence. If no scene transition boundary exists, the coarse localization time point is used as the final cropping boundary.

[0062] The aforementioned dual-layer optimization mechanism ensures both the semantic accuracy of the time points and the visual coherence of the cropped video. Meanwhile, the coarse positioning time point serves as a fault-tolerant design, allowing the system to revert to the coarse positioning time point when there are no scene boundaries within the search window, thus guaranteeing the system's usability across various video content.

[0063] This application guarantees that the cropped video footage is complete and coherent, and users will not experience unpleasant viewing experiences such as "starting from the middle of a scene transition" or "ending during a fade-out," significantly improving user satisfaction with the automatic skip and continuous playback functions.

[0064] The modular architecture design of this application makes the responsibilities of each module clear and the coupling low. The AI ​​model or scene detection algorithm can be upgraded independently, and the accuracy, recall and processing efficiency can be flexibly balanced through parameter tuning.

[0065] This application is applicable to various scenarios in the broadcasting industry, such as broadcast media asset management systems and television program production and broadcasting systems, and is also suitable for video content processing scenarios such as internet video platforms and OTT services. It provides intelligent recognition services for the automated processing of video content such as TV dramas, news programs, and variety shows (e.g., intro and outro trimming, continuous playback optimization, intelligent arrangement), solving technical challenges in the broadcasting industry such as low accuracy of traditional fixed-length trimming, high cost of manual annotation, and poor processing timeliness.

[0066] Application examples in the broadcasting industry: Rapid news program production: Automatically removes intros and outros from news video footage, improving production efficiency and meeting timeliness requirements. Intelligent TV series scheduling: Automatically identifies the opening and ending credits to optimize the seamless transition between multiple episodes; Post-production processing for variety shows: accurately locate the boundaries of the opening and closing credits, and support secondary editing and distribution across multiple platforms; Media asset library content management: Automatically annotates the opening and closing metadata of massive historical programs to improve search efficiency.

[0067] Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

Claims

1. A method for intelligent recognition of video title and ending sequences based on the fusion of AI visual models and scene detection, characterized in that: Includes the following steps: Step 1: Adaptively adjust the candidate detection regions of the beginning and end of the film based on the adaptive candidate region adjustment algorithm to obtain the candidate regions of the beginning and end of the film; Step 2: Extract the opening video segment from the opening candidate area at the set sampling interval, and extract the closing video segment from the closing candidate area at the set sampling interval; Step 3: Use an AI visual language model to perform content understanding on the sampled intro video clips, and identify the type and level of visual features of the intro clips; Step 4: Establish a title sequence detection algorithm with feature weights, perform title sequence detection based on feature levels, and coarsely locate the end time point of the title sequence; Step 5: Use an AI visual language model to analyze the sampled end-credits video clips and identify their features; Step 6: Establish an asymmetric threshold algorithm for opening sequence detection, and perform opening sequence detection based on the segment features of the closing sequence to roughly locate the starting time point of the closing sequence; Step 7: Establish a scene boundary optimization algorithm to determine the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning beginning clip end time point and the coarse positioning end clip start time point.

2. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 1, characterized in that: The method for adaptively adjusting the candidate detection regions of the beginning and end of the video based on the adaptive candidate region adjustment algorithm in step 1 is as follows: Step 11: Determine whether the total duration of the candidate area is not less than the total duration of the video. If yes, adopt the proportional strategy and proceed to step 12. If no, adopt the fixed duration strategy and proceed to step 13. Step 12: Set the opening sequence candidate area = total video duration × 1 / 3, and the closing sequence candidate area = total video duration × 1 / 3; Step 13: Set the opening sequence candidate area to a fixed duration A and the closing sequence candidate area to a fixed duration B.

3. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 1, characterized in that: The method for coarsely locating the end time point of the opening sequence using the feature weighting opening sequence detection algorithm includes the following steps: Step 41: Construct a weighting system for visual features, and assign different reliability weights to features based on their level to obtain high-reliability features, medium-reliability features, and low-reliability features. Step 42: Establish a dynamic termination mechanism for high reliability features to perform in-chip detection; Step 43: Establish a continuous verification mechanism for medium and low reliability characteristics to perform header detection.

4. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 3, characterized in that: The dynamic termination mechanism is to immediately terminate the subsequent header scan when a high reliability feature is detected, and record the time point where the feature is located as the header end time point of coarse positioning.

5. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 3, characterized in that: The continuous confirmation mechanism sets a continuous threshold for medium reliability features and a continuous threshold for low reliability features. The continuous threshold for medium reliability features is greater than the continuous threshold for low reliability features. When the number of continuous confirmations for medium reliability features or low reliability features reaches the continuous threshold for medium reliability features or the continuous threshold for low reliability features, it is determined as the end point of the title sequence, and the coarse positioning title sequence end time point is recorded.

6. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 1, characterized in that: The asymmetric threshold start-of-screen algorithm in step 6, which performs start-of-screen detection based on the segment features of the end-of-screen, and the method for coarsely locating the start time point of the end-of-screen includes the following steps: Step 61: Set up a forward counter and a reverse counter to detect and count the end-of-film features and the main-film features, respectively; Step 62: Establish reset rules: The positive counter increments by 1 when it detects the end-of-film feature and resets to 0 when it detects the main film feature; the negative counter increments by 1 when it detects the main film feature and resets to 0 when it detects the end-of-film feature. Step 63: Set the confirmation threshold for the middle of the end-of-film candidate area, the confirmation threshold for the end of the end-of-film candidate area, and the confirmation threshold for the main film. The confirmation threshold for the middle of the end-of-film candidate area is greater than the confirmation threshold for the end of the end-of-film candidate area. Step 64: Determine whether the segment feature is located in the middle of the end candidate area. If yes, proceed to step 65; otherwise, proceed to step 66. Step 65: When the positive counter continuously reaches the confirmation threshold in the middle of the end-of-film candidate area, it is determined that the end of the film has been reached. The first time point reached by the positive counter is used to coarsely locate the start time point of the end of the film. Step 66: When the forward counter continuously reaches the end confirmation threshold of the end candidate area, it is determined that the end of the film has been reached. The first time point reached by the forward counter is used to coarsely locate the start time point of the end of the film.

7. The intelligent recognition method for video title and ending sequences based on the fusion of AI visual models and scene detection as described in claim 1, characterized in that: Step 7, establishing a scene boundary optimization algorithm, and determining the end clipping boundary of the beginning clip and the start clipping boundary of the end clip based on the coarse positioning beginning clip end time point and the coarse positioning end clip start time point, includes the following steps: Step 71: Set time tolerances before and after the end time of the coarse positioning beginning and the start time of the coarse positioning end; Step 72: Identify the differences between adjacent frames within the time tolerance, and determine whether the differences reach the set threshold. If yes, determine that there is a scene switching boundary and proceed to step 73; otherwise, determine that there is no scene switching boundary and proceed to step 74. Step 73: Calculate the time difference between all scene transition boundaries and the coarse positioning start time point or coarse positioning end time point, and extract the boundary with the smallest time difference as the start time point or end time point. Step 74: Use the boundary where the coarse positioning beginning time point ends or the coarse positioning ending time point begins as the beginning end cropping boundary or the ending start cropping boundary.

8. A video title and end credit intelligent recognition system based on the fusion of AI visual models and scene detection, characterized in that: include Candidate region localization module: Based on the total video duration, adaptively adjust the candidate detection regions at the beginning and end of the video to obtain the candidate regions at the beginning and end of the video. Video sampling module: Extracts the opening video segment from the opening candidate area according to the sampling interval, and extracts the closing video segment from the closing candidate area according to the sampling interval; AI Semantic Analysis Module: Uses AI visual language models to understand the content of sampled intro video clips and identify the types of visual features in the intro of the clips; Feature weight judgment module: Based on the feature types identified by the AI ​​semantic analysis module, different reliability weights are assigned, and a differentiated confirmation strategy is executed based on the differentiated weight features to roughly locate the end time point of the opening sequence. Dynamic Termination Module: Real-time monitoring feature weight judgment module. When a high reliability feature is detected, the subsequent header scan is immediately terminated, and the time point where the feature is located is recorded as the header end time point of coarse positioning. Asymmetric threshold strategy module: Uses AI visual language model to analyze the sampled end-credits video clips to identify clip features, tracks and detects clip features based on a bidirectional counter mechanism, and coarsely locates the start time point of the end-credits video. Scene boundary optimization module: For the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro, the scene detection algorithm is used to locate the scene switching boundary within the time tolerance range, or the end time point of the coarse positioning intro and the start time point of the coarse positioning outtro are used as the scene switching boundary.

9. The intelligent video title and end credit recognition system based on the fusion of AI visual models and scene detection as described in claim 8, characterized in that: The AI ​​semantic analysis module identifies multi-level visual feature types for the opening sequence, including strong semantic markers, fixed visual patterns, and weak stylistic features. Feature levels are set according to the type of visual feature, including first-level features, second-level features, and third-level features.

10. The intelligent video title and end credit recognition system based on the fusion of AI visual models and scene detection as described in claim 9, characterized in that: The feature weight judgment module assigns the first-level feature as a high-reliability feature, the second-level feature as a medium-reliability feature, and the third-level feature as a low-reliability feature. The high-reliability feature uses a single confirmation strategy for header detection, while the medium-reliability feature and the low-reliability feature use a continuous confirmation strategy for header detection.

Images