Key point extraction, playing method and device of video, medium and program product

Abstract

This invention provides a method, device, medium, and program product for extracting key points and playing videos. The method includes: determining the structural pattern information of the target video based on its content description information; determining multiple jump points and point description information corresponding to the target video based on the structural pattern information; setting jump points and point description information for the target video so that when the target video is played on the playback page, in response to a first trigger operation, the nearest jump point is determined based on the point description information, and playback jumps from the current time point to the jump point and resumes. This automatically parses the structural pattern of the video content, improves the accuracy of jump points, and allows for seamless playback between exciting scenes, satisfying users' viewing needs for engaging storylines and improving user experience.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a method for extracting key points from a video, a video playback method based on key points, a video playback system, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of streaming platforms, the consumption model of long-form video content is undergoing a transformation. To bring the highly engaging viewing experience of short videos to long-form video scenarios, video playback platforms have launched the "skip-watch" function. This function, by pre-extracting jump points in the video, provides the ability to jump between adjacent jump points, allowing users to pre-select high-energy moments in the content, thereby improving the user viewing experience and retention rate.

[0003] Current "skip-to-watch" functions, such as those for longer videos like TV series and movies, have a storyline and primarily rely on pre-defined analysis rules to identify skip points. However, video platforms offer a wide variety of videos, including variety shows and educational videos. These videos contain multiple independent structural units, and their viewing scenarios and user needs differ significantly from those of TV series. For example, variety shows offer diverse content styles, such as dating shows, competition shows, and lifestyle experience shows, each with its own unique format. Similarly, educational videos include knowledge-based and technical tutorials, each with its own distinct format. The pre-defined rules are not suitable for matching pre-defined rules and cannot adapt to the diverse and dynamic content of these videos, leading to inaccurate skip point identification and negatively impacting the user's viewing experience.

[0004] Therefore, a technical problem that urgently needs to be solved by those skilled in the art is: how to accurately identify the exciting plot points in a video so that the playback can jump between these exciting plot points. Summary of the Invention

[0005] One objective of this invention is to provide a method for extracting key points from videos, accurately identifying key plot points within the video, enabling seamless playback transitions between these key plot points, and enhancing user interest and experience. The specific technical solution is as follows: In a first aspect of this invention, a method for extracting key points from a video is provided, comprising: determining structural pattern information of the target video based on content description information of the target video, wherein the content description information is a structured description of the summary content of the target video, and the structural pattern information is used to characterize the structural features of the content in the target video; determining multiple jump points and point description information corresponding to the target video based on the structural pattern information; setting jump points and point description information for the target video so that when the target video is played on a playback page, in response to a first trigger operation, the method determines the jump point closest to the current time point based on the point description information, jumps from the current time point to the jump point, and plays the video.

[0006] In a second aspect of this invention, a video playback method based on key points is also provided, comprising: playing a target video on a playback page, the target video having point description information of jump points, the point description information including: a time point corresponding to the jump point, the jump point being obtained based on the structural pattern information of the target video, the structural pattern information being determined based on the content description information of the target video, the content description information being a structured description of the summary content of the target video, and the structural pattern information being used to characterize the structural features of the content in the target video; responding to a first triggering operation, determining the jump point closest to the current time point based on the point description information; and jumping the target video from the current time point to the time point corresponding to the jump point and playing it.

[0007] In a third aspect of this invention, a video playback system is also provided, comprising: a server and a client; wherein, the server determines structural pattern information of the target video based on content description information of the target video, the content description information being a structured description of the summary content of the target video, the structural pattern information being used to characterize the structural features of the content in the target video, and based on the structural pattern information, determines multiple jump points and point description information corresponding to the target video, and sets jump points and point description information for the target video; the client plays the target video on a playback page, and in response to a first trigger operation, determines the jump point closest to the current time point based on the point description information; and jumps the target video from the current time point to the time point corresponding to the jump point and plays it.

[0008] In another aspect of the present invention, an electronic device is also provided, characterized in that it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the program stored in the memory to implement the steps of the video key point extraction method as described in the embodiments of the present invention, and to implement the steps of the video playback method based on key points as described in the embodiments of the present invention.

[0009] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform any of the above-described video key point extraction methods and key point-based video playback methods.

[0010] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above-described video key point extraction methods and key point-based video playback methods.

[0011] The video key point extraction method provided in this embodiment of the invention determines the structural pattern information of the target video based on the content description information of the target video. The content description information is a structured description of the summary content of the target video, and the structural pattern information is used to characterize the structural features of the content in the target video, providing guidance for subsequent analysis of the target video. Then, based on the structural pattern information, multiple jump points and point description information corresponding to the target video are determined, automatically identifying the location of content that attracts users from the target video, and determining the corresponding jump points and point description information, improving the accuracy of point identification. Then, jump points and point description information are set for the target video, so that when users watch the target video, they can jump to the time point corresponding to the jump point to start playback, satisfying users' viewing needs for exciting plots and improving user experience. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0013] Figure 1 This is a flowchart illustrating the steps of an embodiment of the video key point extraction method of the present invention; Figure 2 This is a flowchart of a structural pattern information parsing sub-step in an embodiment of the present invention; Figure 3 This is a flowchart of another structural pattern information parsing sub-step in an embodiment of the present invention; Figure 4 This is a flowchart of a program segment and candidate location identification sub-step in an embodiment of the present invention; Figure 5 This is a flowchart of a program segment cutting step in an embodiment of the present invention; Figure 6 This is a flowchart of another program segment cutting step in an embodiment of the present invention; Figure 7 This is a flowchart of a candidate point identification sub-step in an embodiment of the present invention; Figure 8 This is a flowchart of another candidate point identification sub-step in an embodiment of the present invention; Figure 9 This is a flowchart of a jump point analysis sub-step in an embodiment of the present invention; Figure 10 This is a flowchart of another jump point analysis sub-step in an embodiment of the present invention; Figure 11 This is a flowchart of an optional embodiment of the video key point extraction method of the present invention; Figure 12 A flowchart illustrating the steps of another optional embodiment of the video key point extraction method of the present invention; Figure 13 This is a flowchart illustrating the steps of an embodiment of a video playback method based on key points according to the present invention; Figure 14 This is a schematic diagram illustrating a plot jump example according to an embodiment of the present invention; Figure 15 This is a flowchart illustrating the steps of another embodiment of the video playback method based on key points according to the present invention; Figure 16 This is a schematic diagram of an embodiment of the video playback system of the present invention; Figure 17 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0015] This invention provides a method for extracting key points from videos. Addressing the characteristics of videos—multiple independent structural units, diverse video categories, and rapid updates—this method analyzes and understands the video's content, parsing its structure and highlight types. The video is then segmented, jump points are identified, and inserted into the video for the audience. This eliminates the need for manually defining matching rules, significantly improving video processing efficiency and enhancing the coverage efficiency and user experience of the "jump-to-watch" function.

[0016] Reference Figure 1 The diagram illustrates a flowchart of an embodiment of a video key point extraction method according to the present invention.

[0017] Step 102: Determine the structural pattern information of the target video based on the content description information of the target video.

[0018] The target video is a video played on a video playback platform, which provides the video to users through video playback websites, applications, and apps. The target video is a video with multiple independent structural units, such as variety shows or educational videos. Variety shows are a comprehensive form of television or online video program that integrates various art forms and entertainment elements, aiming to provide viewers with a relaxed and enjoyable viewing experience. Taking variety shows as an example, there are currently many categories of variety shows, and they are updated rapidly. Variety shows have various types, such as: dating, competition, life experience, and role-playing. Even within the same type of variety show, there are variations. For example, role-playing variety shows can be detective shows or escape room shows, while competition variety shows can be singing competitions, sketch comedy competitions, or talk show competitions. This means that general matching rules are not suitable for variety shows; instead, different parsing and matching rules must be developed for different variety shows to accurately locate the key points of the plot.

[0019] Content description information is a structured description of the target video's summary content. This summary is divided into structured elements such as name, main theme, format, participating users, and plot details, typically ranging from 1000 to 3000 words. For example, the content description information for variety show videos includes descriptions of the program name, theme, format, participating guests, and main plot points. The content description information for educational videos includes descriptions of the knowledge points (or technical points), principles, demonstrations, application clustering, and summaries. Based on this content description information, the target video's content structure is analyzed to reveal its structural pattern information. This structural pattern information characterizes the structural features of the video's content, including the type of content and program segments.

[0020] Step 104: Based on the structural pattern information, determine multiple jump points and point description information corresponding to the target video.

[0021] Structural pattern information can determine the structural features of the content in a target video. Based on this information, the content can be identified, and through semantic understanding and other processing, multiple jump points and their descriptions can be determined. The descriptions include time points corresponding to the target video segments, such as the start time point. The start time point of exciting video segments is added to the descriptions to locate the target video segment.

[0022] Step 106: Set jump points and point description information for the target video so that when the target video is played on the playback page, in response to the first trigger operation, the jump point closest to the current time point is determined based on the point description information, and the video is played from the current time point to the nearest jump point.

[0023] After extracting the location description information, this information can be associated with the target video. This involves setting jump points and their descriptions for the target video. For example, jump points can be set in the target video according to their corresponding time points. Alternatively, the location description information can be bound to a predetermined operation, allowing playback to jump from the current time point to the next jump point based on that operation. Thus, during video playback, upon receiving a predetermined operation, playback can jump to the next time point, starting from that point. This allows users to skip to the next exciting part of the story if they are not interested in the current plot, satisfying their viewing needs and improving the user experience.

[0024] In summary, based on the content description information of the target video, the structural pattern information of the target video is determined, providing guidance for subsequent analysis of the target video. Then, based on the structural pattern information, multiple jump points and their descriptions are determined for the target video. The locations of the jump points containing content that attracts users are automatically identified from the target video, improving the accuracy of the jump points and their descriptions. Finally, jump points and their descriptions are set for the target video, so that when users watch the target video, they can jump to the time point corresponding to the jump point to start playback, satisfying users' viewing needs for exciting plots and improving user experience.

[0025] This invention shifts from a "rule-driven" approach to a "cognitive-driven" approach, automatically analyzing program feature patterns by performing semantic understanding and structural parsing on the content description information of video content. The content description information includes the program name and a brief summary, which includes the theme, summary content (such as format, participating users, and plot details). The structural pattern information includes type information, jump duration range, and program segments. The type information includes program type and jump content type. Program type refers to the type of program content; for example, variety show types include music, cultural tourism, comedy, emotional, gaming, talk shows, lifestyle services, and role-playing. Other classification methods can also be used, and this invention does not limit this. The jump duration range is the duration range for setting jump points in the video. Program segments indicate nodes in different content segments of the video. Jump content type refers to the type of engaging plot, storyline, or other content in the video.

[0026] In one alternative embodiment, such as Figure 2 As shown, step 102, which determines the structural pattern information of the target video based on the content description information of the target video, includes the following sub-steps: Sub-step 202: Based on the program name and content description, determine the program type and jump duration range of the target video.

[0027] Semantic understanding processing is performed on the content description information of the target video. By performing semantic analysis on the content description information and combining it with the context, the structural pattern information of the target video is determined. This includes determining the program type, jump duration range and program segment of the target video, as well as one or more jump content types in each program segment, through semantic analysis and context.

[0028] This process involves semantic analysis of the content description information. This can be achieved through various methods, such as semantic analysis based on relevant rules combined with dictionary analysis, semantic analysis using knowledge graphs, or semantic analysis based on machine learning and deep learning. This helps determine the program positioning and type of the variety show video. For example, it involves comprehensively analyzing the program's name, theme, format, and main plot to determine the corresponding program type. Program positioning refers to pre-planned data from the program producers regarding the video's ideological content, target audience, program style, and production style. The pacing information of the variety show is analyzed in conjunction with the program type to determine the jump duration range. For example, comedy, game shows, and role-playing shows typically have a faster pace and can have a dense jump duration range. Conversely, cultural tourism, talk shows, and lifestyle service shows typically have a slower pace and can have a sparse jump duration range. The appropriate jump duration range is determined by comprehensively considering the program type. Dense and sparse are relative terms and can be set based on specific needs; this embodiment of the invention does not impose any limitations on this.

[0029] Sub-step 204: Based on the content summary, determine the program segments and jump content types of the target video.

[0030] Semantic analysis is performed on the content description to determine the structural form of the variety show video. This structural form represents the program's segments. Then, the transition points between segments are analyzed to identify the corresponding program segments. Taking game competition shows as an example, they typically have multiple game segments. These segments can be divided into segments, with corresponding program segments set up, such as based on location changes or rule announcements. Another example is role-playing mystery variety shows, which include multiple segments such as case introduction, evidence gathering, discussion, voting, and revealing the truth. Corresponding program segments are set up based on these segments.

[0031] Furthermore, semantic analysis can be used to identify the highlights of a variety show video, which refers to the engaging plot content. Taking variety shows as an example, the compelling plots differ across genres. For instance, emotional variety shows typically feature interactions and emotional exchanges between guests, game shows highlight moments and humorous scenes, and mystery variety shows involve plot progression and the search for crucial evidence. Therefore, based on content description information, the highlights of different types of programs can be analyzed and categorized. Furthermore, different types of content can be identified within different program segments.

[0032] In another parallel implementation, a pre-built language model can be used to automatically analyze the content structure of the target video and determine the corresponding structural pattern information, thereby improving processing efficiency. Here, the language model is an abstract mathematical model built based on objective linguistic facts and is a fundamental tool in the field of computational linguistics. Natural Language Processing (NLP) is performed based on the language model to understand the semantics of the video description information, thereby extracting key plot information from the video clips. The pre-built language model can be various NLP-based language models, such as Transformer-based NLP models, multi-task learning models, neural network models, and Large Language Models (LLMs). The LLM model is a deep learning model trained using a large amount of text data, enabling it to generate natural language text or understand the meaning of language text.

[0033] Input data is constructed based on content description information, and then fed into a pre-built language model for analysis and processing to output structural pattern information of the target video. In one example, the key plot information also includes the reasons for selecting each parameter in the structural pattern information.

[0034] In this embodiment of the invention, the preset language model can be optimized in real time. Feedback information can be determined by combining the user's operation during playback, or a feedback interface can be provided to the user to obtain the user's feedback information. Based on the feedback information, the preset language model and prompt word template can be optimized to improve the accuracy of model processing.

[0035] In some embodiments, for pre-built language models such as LLMs, cue word engineering can be used to guide the model's processing. Cue word engineering designs and optimizes cue words to guide the large language model in generating output that meets specific needs.

[0036] In another alternative embodiment, such as Figure 3 As shown, step 102, which involves parsing the content structure of the target video based on the content description information to determine the structural pattern information of the target video, includes: Sub-step 302: Obtain the first prompt word template, which includes variable parameters and prompt items. The prompt items include at least one of the following: analysis content and output requirements.

[0037] In this embodiment of the invention, the processing capability of the pre-set language model is used to automatically analyze the structural patterns of the content in the video. Therefore, when setting the prompt word template, the large language model should be guided to output structured analysis results. Professional role-playing prompt words are designed, which include the following key elements: (1) Role positioning: The large language model is positioned as a "variety show producer who focuses on content operation", so that it can analyze from the perspective of professional content understanding, rather than simple text processing. (2) Task description: Clearly inform that the purpose of the analysis is to serve the "skip" function, and require the model to understand the core requirement of "quickly locating wonderful segments while maintaining coherent understanding". (3) Three-dimensional analysis framework: Guide the model to conduct structured analysis from three dimensions: Program positioning: Identify program type (competition / observation / travel / reasoning, etc.) and unique highlights; Structural analysis: Find the key nodes that are suitable as the division of large segments (rule introduction / segment transition / result announcement, etc.); Highlight mining: Locate the type of wonderful moments that are most likely to resonate with users (emotional outburst / plot reversal / team interaction, etc.); (4) Format constraints: Require the model to strictly return the specified format, such as JSON format, to ensure that the subsequent program can automatically parse it.

[0038] Based on the above design concept, a first prompt template is set up. The prompt template includes variable parameters and prompt items. The prompt items include at least one of the following: analysis content and output requirements. The analysis content refers to the key information to be analyzed, including program positioning, program structure, and highlight identification. The variable parameters include: a name variable and a description variable. The name variable corresponds to the program name, and the description variable corresponds to a brief summary of the content.

[0039] Here is an example of a first prompt word template: You are a variety show producer focused on content operation, and you need to provide professional advice on the "skip-watch" feature. The skip-watch feature aims to help users quickly locate exciting segments of the show while maintaining a coherent understanding.

[0040] Based on your professional experience, please analyze the following input information and provide strategic guidance for program segmentation: 1. Program Positioning: Identify the program type (such as competition / observation / travel / reasoning, etc.) and its unique highlights; 2. Structural Analysis: Identify key nodes suitable as boundaries between large sections (such as rule introductions / stage transitions / result announcements, etc.); 3. Highlights: Identify the most relatable and engaging moments for users (such as emotional outbursts, plot twists, team interactions, etc.); ==== Enter Information ==== <Program Name> {{show_name}} <Synopsis> {{epis_desc}} ==== Return Format ==== ATTENTION: ONLY RETURN VALID JSON WITH ```json BEGINING. DO NOTINCLUDE ANY OTHER TEXT. json { "type": "Reasoning", / / Core type tag "segment_duration_range": [240, 480], / / Recommended jump duration range (seconds) to balance jump efficiency and content continuity "coarse_actions": [ / / Signature actions / scenes during transitions] "Case Introduction", "On-site Evidence Collection", "Focused Discussion", "Vote Announcement", "Truth Revealed" ], "highlight_categories":{ / / Note the output type, not the specific event.} "common": { / / General high-energy point type "personality": ["Talent Showcase", "Quick Responses", "Leadership Demonstration", "Personal Breakthrough"], / / Charismatic Personality Demonstration "emotion": ["breakdown and sobbing", "out of control with anger", "frozen in shock", "dancing wildly with ecstasy"], / / emotional outburst "interaction": ["team spirit erupts", "tacit cooperation", "fierce competition", "warming up of feelings"], / / interpersonal interaction }, "specific": { / / Track-specific high-energy point types (for example, reasoning-based games) "plot": ["Unexpected Discovery", "Twist of Truth", "Conspiracy Revealed", "Misunderstanding Cleared Up"], / / Plot Twist "reasoning": ["connecting clues", "logical analysis", "questioning evidence", "key to solving the case"], / / reasoning process } } "reason": "" / / Note your reason or inspiration } ``` 3. Output Result Structure The JSON data returned by the large language model contains the following key fields: Field 1: Program type (i.e., the core type label of the program); Purpose: to quickly identify the program's category; Example values: "reasoning", "romance", "competition", "travel"; Use: to provide type context for subsequent processing. Field 2: Jump duration range (segment_duration_range); Purpose: to guide the setting of jump density; Format: [minimum duration, maximum duration], in seconds; Example value: [240, 480] indicates that it is recommended to set a jump point every 240-480 seconds; Design intention: different types of programs have different rhythms. For example, fast-paced competition programs may need more dense jump points (180-300 seconds), while slow-paced observation programs may need more sparse jump points (300-600 seconds). Field 3: Program Segment Coarse Actions (i.e., the iconic actions / scenes of switching between different segments in the program); Purpose: Defines the main segment types of the program, providing a basis for identification in subsequent coarse-grained segmentation; Format: String array, Example values: ["Case Introduction", "On-site Evidence Collection", "Focused Discussion", "Vote Announcement", "Truth Revealed"]; Use: These segment names will be used as identification tags during sliding window detection. Field 4: Jump Content Type Highlight Categories (i.e., the type system of exciting plot points and other key moments in the program); Purpose: Constructs the classification framework of the program's highlights, providing a selection standard for subsequent refined positioning; Structure: Divided into two levels of classification: Common and Specific; Common: Universal highlight types applicable to all variety shows; Specific: Highlight types unique to this category.

[0041] In this embodiment of the invention, the jump content types include: general types and specific types. General types are jump content types applicable to various variety shows, such as guest introductions, emotional outbursts, and guest interactions. Specific types are the unique and exciting storylines specific to the variety show video; for example, specific types for mystery-solving shows include mystery elements and plot twists.

[0042] This allows for the construction of a suitable first prompt word template, prompting the pre-built language model to perform analysis and processing for the skip function.

[0043] Sub-step 304: Combine the content description information of the target video with the first prompt word template to generate the first input information.

[0044] By inputting episode summaries (including program name, theme, guests, plot summary, and other textual information) into a large language model, the model's semantic understanding capabilities are used to automatically extract the program's structural features and highlight types.

[0045] Sub-step 306: Input the first input information into a preset language model for semantic analysis to obtain the program type, jump duration range, program segment, and jump content type of the output target video.

[0046] The first input information is input into a pre-set language model for semantic analysis, including analysis of program positioning, content structure, highlights, etc., to determine the program type, jump duration range, program segment and jump content type of the target video as structural pattern information.

[0047] The embodiments of the present invention can realize automatic knowledge transfer. The large language model has learned a large amount of variety show knowledge in the pre-training stage and can automatically transfer the existing knowledge to the analysis of new programs without the need for manual rule summarization.

[0048] To achieve structured output, a carefully designed JSON format constraint is used to ensure that the model output can be directly parsed and used by the program, avoiding the ambiguity of natural language.

[0049] The system establishes a hierarchical identification mechanism for compelling storylines, employing a two-tiered classification design that combines "general" and "exclusive" categories. This approach ensures consistency across programs (general categories) while preserving the unique characteristics of each program (exclusive categories).

[0050] The adaptive jump point density setting, through the segment_duration_range field, allows the model to automatically recommend appropriate point density based on the program rhythm, avoiding a "one-size-fits-all" fixed interval.

[0051] This invention, based on a large-scale model for automatic program structure understanding, utilizes a large language model to automatically analyze variety shows, identifying structural patterns such as program type and highlights, thus eliminating the need for traditional methods that require manual pre-definition of program structure. On one hand, it achieves cross-category universal processing, eliminating the need to develop separate rules for each variety show type, significantly reducing the adaptation cost for new categories. On the other hand, it solves the problem of insufficient accuracy in manual processing due to a lack of professional variety show production knowledge. By leveraging the extensive knowledge reserves and understanding capabilities of the large-scale model, it can more accurately grasp the inherent rules and structural characteristics of different variety show categories, improving the professionalism and reliability of jump point identification.

[0052] The embodiments of the present invention are based on an adaptive mechanism. First, the overall structure of the program is understood, and then specific jump points are detected under the guidance of the structure.

[0053] The identification of jump points in a target video in this embodiment of the invention can be divided into two stages: stage 1 is a coarse-grained segmentation stage, and stage 2 is a fine-grained identification stage. In an optional embodiment, step 104, based on the structural pattern information, determines multiple jump points and point description information corresponding to the target video, including: Based on the structural pattern information, the target video is identified to determine multiple candidate locations; The multiple candidate locations are filtered to determine the corresponding multiple jump points and location description information.

[0054] Based on this structural pattern information, the content of the target video is identified. This identification can be performed in multiple steps. For example, the target video is first segmented into multiple video clips, and then semantic understanding is performed on each video clip to identify multiple candidate points and their location information. Alternatively, the descriptive information of the video clips is processed through word segmentation, and then combined with contextual information to determine semantic content and keywords, thereby identifying candidate points and their information. Candidate points are the time points corresponding to key content, and the location information is the descriptive information of the corresponding key content. Key content refers to attractive plots, storylines, key points, etc., in the video. Then, the candidate points are sorted chronologically. When the time interval between two adjacent candidate points is small, the two points can be merged or the point with higher appeal can be retained, resulting in multiple jump points and their descriptive information.

[0055] In one alternative embodiment, such as Figure 4 As shown, based on the structural pattern information, multiple candidate points in the target video are determined, including: Sub-step 402: Based on the program segments, the target video is segmented to determine multiple video segments.

[0056] The program segments are matched within the target video. Once a segment is matched, it is used as a segment segmentation point for further segmentation. By matching the identifiers of each segment segmentation point, the target video is divided into multiple video segments. The target video can be described using video description information, which describes the plot content of the video segments. This video description information can be obtained by identifying the video data corresponding to the video segments, including the identification of video images and audio data. Alternatively, the video description information can be determined in conjunction with plot information provided by the target video's producer; however, this embodiment of the invention does not impose limitations on this approach.

[0057] The video description information is matched with the program segments, and the matched program segments are used as segmentation points for segmentation.

[0058] Sub-step 404: Based on the audio-visual content of each video segment, determine multiple candidate points and point information that match the jump content type.

[0059] After segmenting the video into segments corresponding to each program, the audio-visual content of each video segment is analyzed in conjunction with the type of content to be jumped to, and the plot content in the video segment is analyzed in depth to screen candidate points and determine point information, providing a data foundation for the subsequent selection of jump points.

[0060] In one optional embodiment, sub-step 402, which involves segmenting the target video based on the program segment to determine multiple video segments, includes: obtaining video description information of the target video and using a sliding time window to detect whether the video description information contains the program segment; and segmenting the target video into multiple video segments based on the detected program segment.

[0061] To improve processing efficiency, the video description information is used to describe the plot content of the target video for a predetermined duration. The video description information can be obtained by identifying video data, including identifying video images and audio data. It can also be determined in conjunction with plot information provided by the target video's producer; however, this embodiment of the invention does not impose limitations on this approach.

[0062] The description of a predetermined duration of plot content can be divided into multiple plot units. For example, a video clip can be divided into plot units of 15-20 seconds each. Each plot unit describes the plot from multiple dimensions, including visuals and audio, thus refining the video clip into individual plot units. This detailed information organization provides a data foundation for subsequent analysis, improving the accuracy and comprehensiveness of the analysis. Each plot unit corresponds to time information and audio-visual content. The time information of a plot unit can be the time range of the plot segment or a specific time point. The time point can be the start time of the plot segment or other specified time points within the plot segment, such as the time points containing characters. In one optional embodiment, the time point of the plot unit is a timestamp. The plot units can be sorted according to their time points in the video description information, that is, the video description information includes a sequence of time points for the plot units, with each time point in the sequence corresponding to a plot unit, and each plot unit described through audio-visual content.

[0063] The audio-visual content refers to the visual and audio content presented in the story unit. By recognizing each video frame and audio data in the story unit, the content presented on the screen can be identified, such as the characters, environment, objects, and the relationships between them. The content presented on the audio can also be identified, such as the dialogue between characters in the story unit, as well as background sounds, interludes, and other sound effects.

[0064] Therefore, each time a sliding time window is used, the video description information corresponding to the predetermined duration is matched with the program segment. If a program segment is included, the video is segmented based on that segment, or by analyzing the content before and after it and setting appropriate segmentation points. If no program segment is included, the matching process continues for the next video description information. This process is repeated to segment the target video into multiple video segments.

[0065] In other alternative embodiments, to facilitate the identification of segmentation points, a sliding window approach can be used to perform segmentation, identifying segment boundaries throughout the entire video. A large language model is then used to analyze the video content description to determine whether there are substantial transitions between major program segments.

[0066] In one alternative embodiment, such as Figure 5 As shown, the method of detecting whether the video description information contains the program segment using a sliding time window includes: Sub-step 502: Obtain the sliding time window.

[0067] The sliding time window has a fixed window duration (window_time), such as 600 seconds (10 minutes). The window duration needs to balance the completeness and accuracy of the process; it needs to be long enough to observe all the features of the process, but not so long that it misses switching points. The sliding time window also has a stride, which is a fixed sliding distance of the time window, such as 30 seconds. The stride needs to be set to strike a balance between detection accuracy and computational cost. The sliding time window starts at the start time of the first clip and ends at the end time of the last clip. In this example, the video description information is the complete target video.

[0068] Sub-step 504: Detect whether the video description information within the sliding time window includes the program segment.

[0069] The video description information can be divided into plot units, with each 15-20 seconds segment representing a plot unit. This allows for a detailed description of the video's plot from the perspective of these plot units. This meticulous information organization provides a data foundation for subsequent analysis, improving its accuracy and comprehensiveness. The video description information includes unit descriptions for each plot unit, used to describe the plot contained within the video clip. The plot can be described from multiple dimensions, such as time, visuals, dialogue, and action. Each plot unit's unit description information includes: unit start time (start_time), unit end time (end_time), and a textual description of the plot content (e.g., "Six guests sat around a conference table, the detective presided over the speech, and everyone earnestly discussed the case").

[0070] By sliding a time window within the video description information, the unit description information of each plot unit falling within the sliding time window is detected to determine whether it contains program segments.

[0071] If not, i.e. the program segment is not included, proceed to sub-step 506; if not, i.e. the program segment is included, proceed to sub-step 508. Sub-step 506: Continue detection by sliding the sliding time window backward for a first duration.

[0072] The current sliding time window does not contain any program segments. The sliding time window will be slid in increments of the first duration, and the description information of the next video segment within the sliding time window will be matched.

[0073] Sub-step 508: Slide the sliding time window backward for a second duration and continue detection, where the second duration is longer than the first duration.

[0074] If the current sliding time window contains program segments, the segment splitting point is determined based on the matched program segments. This segment splitting point is the position where two different segments are separated, and the corresponding timestamps can be used as the point information of the segment splitting point.

[0075] In coarse-grained segmentation, the video is divided into multiple segments. Therefore, when matching one segment, it's usually not easy to match another. In this case, the sliding time window can be moved forward in steps of a second duration, which is longer than the first duration. In some examples, the second duration is longer than the duration of the sliding time window. This can improve the detection speed.

[0076] Sub-step 510: The target video is divided into multiple video segments based on the detected segmentation points.

[0077] Based on the location information determined for each program segment, the target video is divided into multiple video segments, and the content of each video segment is the content of one segment of the program.

[0078] In other examples, coarse-grained segmentation of the target video can also be achieved based on a pre-built language model.

[0079] Input data is constructed based on program segments and video description information. This input data is fed into a pre-built language model for segmentation point detection, outputting segmentation information for the target video. In one example, the segmentation information includes: timestamps of the segmentation points, segments before and after the segmentation, and the reason for the segmentation. The segments before and after the segmentation refer to the segments before and after the switch, and the reason for the segmentation is the specific basis for determining a switch point. In some examples, parameters such as the program name can also be added when constructing the input data to specify the program being analyzed.

[0080] Another parallel implementation, such as Figure 6 As shown, sub-step 402 involves segmenting the target video based on the program segment to determine multiple video segments, including: Sub-step 602: Obtain the second prompt word template. The prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0081] The second prompt template is used to indicate the segmentation process of the video. Variable parameters include: segment variables and description variables. The segment variables are used to fill in program segments, and the description variables are used to fill in video description information. It may also include a name variable to fill in the program name, thereby clearly identifying the program being processed in the video.

[0082] The task content for the second prompt word template is designed based on the following elements: (1) Role positioning: The large language model is positioned as a "professional video editor", so that it can judge the segmentation point from the professional perspective of video editing, rather than simple text classification. (2) Task requirements: The model is required to identify the "segmentation point" and provide the "segment before conversion", "segment after conversion" and "judgment basis", etc. (3) Judgment Criteria Emphasis: The three core judgment criteria are clearly conveyed through prompts to avoid the model misjudging small scene transitions within a segment as major segment transitions: Criterion 1: Substantial progress of the program; Positive example: from "Introduction of Task Rules" to "The Task Officially Begins"; Negative example: in the same task, from "Indoor Scene" to "Outdoor Scene" (which is a scene transition, not a segment transition); Criterion 2: Change of core objective; Positive example: from "Collecting Clues" to "Focusing on Case-Solving Discussion" (the objective changes from obtaining information to analyzing information); Negative example: in the evidence search segment, from "Searching Room A" to "Searching Room B" (the objective is still to search for evidence); Criterion 3: Switching of complete plot units; Note: Major segments usually last for a long time and contain multiple related scenes; Avoid: misjudging the internal shot transitions as segment transitions.

[0083] The model output can be designed based on the following principles: The large language model returns input in a predefined format, such as a JSON array, where each element represents a segmentation point of a detected segment. Field 1: idx (segmentation point number); Purpose: To number multiple segmentation points detected within the same window; Format: Integer starting from 0. Field 2: timestamp (time point of the segmentation point, such as a timestamp); Format: Floating-point number, in seconds; Constraint: Must be within the range of [current_time, window_end_time] of the current window; Example: 1250.5 indicates that the segment switch occurs at the 1250.5th second of the video. Field 3: from (segment before the switch); Format: String; Example: "On-site evidence search"; Note: Predefined segment types can be used, or segment names summarized by the model itself can be used. Field 4: to (segment after the switch); Format: String; Example: "Group discussion"; Note: Flexible expression is also allowed. Field 5: reason (segmentation reason); Format: string, natural language description; Example: "The scene switches from a scattered evidence-gathering scene to six guests sitting around discussing, the detective announces the end of the evidence gathering, and the discussion begins"; Purpose: Provides interpretability, facilitating subsequent quality assessment and debugging.

[0084] Based on the above elements, a second prompt word template is designed so that the input data constructed based on the second prompt word template can guide the model to segment the program into multiple video segments.

[0085] The template for the second prompt word is as follows: You are a professional video editor, and you need to help identify the segment cuts in a variety show video. I will provide the following information: 1. Program title; 2. Pre-defined list of program segments (coarse_actions): These are the main segment types pre-defined by the program team, but may not be complete; 3. Description of the current time window (window_content): a text description of the video content every 15 seconds; Your task is to analyze the window content during this period and determine: 1. Are there any points where the process is divided (if so, please specify the exact time and the basis for the transition)? 2. Explain the process before and after the conversion for each segmentation point; ## Key to Judging Major Transitions: - Substantial progress in the program's development (such as transitioning from task introductions to the official start); - A shift in the core objective (e.g., from gathering clues to discussing how to solve the case); - Switching between complete story segments (rather than switching between small segments within the same segment); ## Notes: - Focusing only on the transformation at the major level, ignoring the scene switching within the process; - Clear evidence of substantial changes in the process needs to be observed; - A major segment typically lasts a long time and includes multiple related scenarios; -timestamp must be the actual time point that appears in window_content; - If no clear major transition is observed in the current window, you can return to an empty list and do not force a switch point; ==== Return Format ==== ATTENTION: ONLY RETURN VALID JSON WITH ```json BEGINING. DO NOTINCLUDE ANY OTHER TEXT. json [ { "idx": 0, "timestamp": The detected switching time (float type in seconds). "from": "The steps before the switch", "to": "The switched stage", / / The "to" and "from" fields can supplement stage types not included in the preset stage list. "reason": "The specific basis for determining a switching point" } ] ``` ==== Enter Information ==== <Program Name> {{show_name}} / / Note: This section fills in the blanks for different TV series, and the combined string is then fed into the larger model. <Preset Stage Type> {{coarse_actions}} <Current time window content> {{window_content}} This leads to the construction of a second prompt word template to guide the model's processing. A sliding window method is employed to identify segment boundaries throughout the video. A large language model is used to analyze the video content description and determine whether there are substantial transitions between major program segments.

[0086] Sub-step 604: Combine the video description information, program segment, and the second prompt word template to generate the second input information.

[0087] In constructing the input and output, in one example, all video description information and program segments can be filled into the variable parameter values ​​of the second cue word template to generate the second data information. In another example, video description information and program segments within a sliding time window can be obtained and filled into the variable parameter values ​​of the second cue word template to generate the second data information. Providing video description information within the sliding time window each time reduces the amount of data processed by the model. When detecting segment switching points, the model can also move according to the second duration, thereby further reducing the amount of data processed and improving processing efficiency.

[0088] An example input is: Input example: Program Title: "Detective Variety Show" Preset stage types: ["Case introduction", "On-site evidence collection", "Group discussion", "Vote announcement", "Truth revealed"] Example of a window content snippet: [ { "start_time": 1200.0, "end_time": 1215.0, Description: "A carefully searched the drawers in the study and found a letter." }, { "start_time": 1215.0, "end_time": 1230.0, Description: "B is inspecting the tableware in the kitchen, looking nervous." }, / / ... intermediate records omitted { "start_time": 1485.0, "end_time": 1500.0, "description": "Detective C gathers everyone and announces the end of the evidence collection period." }, { "start_time": 1500.0, "end_time": 1515.0, Description: "Six guests sat around the conference table and began discussing the clues they had each discovered." } ].

[0089] Sub-step 606: Input the second input information into a preset language model for segment detection to obtain the segmentation information of the output target video. The segmentation information includes: timestamps of segmentation points, segments before and after segmentation, and the reason for segmentation.

[0090] The second input information is input into a preset language model for segment detection, and the segmentation information of the target video is output, including the timestamp of the segment segmentation point, the program segment before segmentation, the program segment after segmentation, and the reason for determining the segment segmentation point.

[0091] One example of output: [ { "idx": 0, "timestamp": 1485.0, "from": "On-site evidence collection", "to": "Focus on discussion", "reason": "Detective A clearly announced the end of the evidence collection, and the scene switched from the guests being scattered in various rooms collecting evidence to everyone sitting around a conference table discussing. The objective of the segment shifted from collecting evidence to analyzing clues, which is a significant change in the segment's focus." } ].

[0092] Once the segmentation information is determined, the target video can be switched into multiple video segments based on this information.

[0093] After coarsely segmenting the target video to determine the video segments corresponding to each stage, the jump points are further identified to ensure the smoothness and attractiveness of the user's viewing experience.

[0094] In one alternative embodiment, such as Figure 7 As shown, sub-step 404, based on the audio-visual content of each video segment, determines multiple candidate points and point information that match the jump content type, including: Sub-step 702: Determine plot information based on the video description information of the target video.

[0095] To improve the accuracy of plot positioning, the time point of each plot unit can be accurate to two decimal places, such as 243.28, 257.24, etc. The descriptive information of the plot unit includes at least one of the following: scene description information, character interaction information, and action description information. The scene description information is used to describe the plot unit from the perspective of the scene's location, for example, by identifying each video frame in the plot unit to determine the described scene content. This could describe a scene such as a forest or city, containing several characters, props, and camera transitions. Character interaction information is used to describe the plot segment from the perspective of the dialogue between the participating guests, including the guests in the segment and their dialogue. Action description information is used to describe the plot segment from the perspective of character actions, including descriptions of the actions of the guests in the plot segment and their interactions. The video description information also includes a summary of the program segment to which it belongs, which describes the summary plot content of the segment, helping to understand the content format and specific plot development of the segment.

[0096] By performing semantic understanding on the video description information and analyzing the plot content corresponding to each plot unit, the key plot points in that segment are identified, generating plot information. Plot information includes the time frame and content of the plot. The plot content describes the specific details of the plot segment and its role in that segment.

[0097] Sub-step 704: Classify the plot information. When the classification result of the plot information is a jump content type, determine the corresponding candidate location and location information.

[0098] The identified plot information is matched and analyzed using the jump content type to determine whether the plot content matches the jump content type. If a match is found, candidate points and point information are determined based on the plot information. Specifically, the start time of the plot can be set as the switching time of the candidate point, and the switching time, the corresponding jump content type, and the plot content are used as the point information.

[0099] In another parallel implementation, refined identification of candidate points can also be achieved based on a pre-built language model. Input data is constructed based on video description information and the type of content to be switched to. This input data is then fed into the pre-built language model for plot content detection, outputting the point information of candidate points. In one example, the segmentation information includes: the timestamp of the candidate point, the type of content to be switched to, and the reason for selection. The reason for selection serves as the specific basis for selecting candidate switching points. In some examples, parameters such as the program name can also be added when constructing the input data to clarify the program being analyzed and maintain consistency in model processing.

[0100] The pre-built language model requires the following input data: Highlight categories, which are determined during the structural pattern recognition of the video content and include common and specific categories. For example, common categories include: intelligent, acting prowess, tension, sudden realization, heated debate, and seamless cooperation. Specific categories include: discovery of key clues, exploitation of rule loopholes, identity probing, and discovery of contradictory evidence. Video description information is presented in chronological order according to the storyline units, including start time, end time, and description information such as scene description, character behavior, and key dialogue. Input information may also include the program name to ensure consistency in the pre-built language model's processing.

[0101] In one alternative embodiment, such as Figure 8 As shown, sub-step 404, based on the audio-visual content of each video segment, determines multiple candidate points and point information that match the jump content type, including: Sub-step 802: Obtain the third prompt word template. The third prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0102] The variable parameters include: plot parameters and type parameters. Plot parameters are used to populate video description information, facilitating subsequent understanding and analysis of the plot. Type parameters are used to populate the type of content to be jumped to, helping the model determine plot content that matches that type. The task content refers to the processing tasks required to identify candidate points for compelling plot points. The output requirements define the format and content of the output data.

[0103] When setting up the third prompt template, the following points can be considered: Role Positioning: Position the large language model as a "professional variety show director," enabling it to judge which moments are worthy of user attention from a professional content production perspective. Clear Task: Require the model to identify key moments in video clip descriptions that match predefined high-energy point types and explain the reasons for its judgment. Multi-dimensional Judgment Criteria: Convey the core judgment dimensions through prompts to ensure that the selected high-energy points have genuine viewing value.

[0104] An example of a third-party prompt template is as follows: As a professional variety show director, you will receive two key inputs related to the program "{{show_name}}": 1. `highlight_categories`: Predefined categories for the types of content that can be redirected, including: - common: General types of content for navigation (such as character performance, emotional reactions, interactive effects, etc.) - Specific: Program-specific content types (such as plot development, mystery progression, etc.) 2. seg_clip_data: Unit description information for story segments arranged in chronological order. Each segment contains: - Timestamp - Scene Description - Character behavior - Key Conversation Task requirements: 1. Analyze each time segment to identify key moments that match the high-energy type. Judgment criteria: - Plays a crucial role in driving the plot forward - Able to evoke strong emotional resonance in the audience - Featuring prominent dramatic conflict or suspense - Key interactions that reveal character traits or relationships - Revealing information or clues 2. Precautions: - Avoid overly dense marking; maintain a sense of rhythm. - Focus on key moments of scene transition and atmosphere change. - Pay attention to nonverbal cues such as facial expressions and actions. - Consider contextual coherence ==== Return Format ==== ATTENTION: ONLY RETURN VALID JSON WITH ```json BEGINING. DO NOTINCLUDE ANY OTHER TEXT. json [ { "timestamp": a point in time (float, seconds). "type": "The corresponding high-energy type (from highlight_categories)", "reason": "Specific reasons for identifying a high-energy point, including key actions / dialogue / plot developments." } ] ``` Notice: - The timestamp must correspond to the actual time point in the input segment. - type must be a type defined in highlight_categories. - The reason needs to concisely explain the basis for the judgment. ==== Enter Information ==== <High Energy Point Type Definition> {{highlight_categories}} <Video clip description> {{seg_clip_data}} This allows for the construction of a third prompt word template to guide subsequent processing by a pre-built language model.

[0105] Sub-step 804: Based on the video description information, the type of content to be jumped to, and the third prompt word template, the third input information is generated by combining them.

[0106] The video description information and the type of content to be redirected are filled into the variable parameters of the third prompt word template. Parameters such as the program name can also be added to the variable parameters to generate the third input data. The input video description information can be the unit description information for each story segment within a given segment.

[0107] Sub-step 806: Input the third input information into the preset language model for plot understanding and obtain the point information of the output candidate points. The point information includes: the timestamp of the candidate point, the type of content to which it belongs, and the reason for filtering.

[0108] The third input information is input into a pre-set language model for plot understanding, determining the plot content corresponding to each plot unit, and analyzing whether the plot content belongs to the jump content type. If it belongs to the jump content type, the starting position of the plot can be traced back to determine the timestamp of the corresponding candidate point, the jump content type, and the selection reason, etc.

[0109] In one example of output data, the large language model returns output results in a predefined format, such as a JSON array, where each element represents a candidate point. Input: [ { "timestamp": 418.92, "type": "common.interaction", "reason": "A and B played on the homophones 'AB' and 'BA', creating a fun interaction and showcasing the chemistry between the guests." }, { "timestamp": 436.8, "type": "common.emotion", "reason": "The scene of the NPC character rolling off the upper bunk and being pinned to the ground is visually striking, directly demonstrating the consequences of disobedience and creating a tense and exciting atmosphere." }, ... ] This allows for the automatic identification and recommendation of candidate locations within each stage, facilitating subsequent filtering and insertion.

[0110] This invention implements an intelligent prediction mechanism for jump content types. Based on the characteristics of program content patterns, it can automatically predict the types of jump content that may occur in each stage using a large model, rather than relying on human experience. This improves the accuracy and completeness of jump point identification and can discover potential high-energy moments that may be missed by manual rules.

[0111] In some optional embodiments of the present invention, some manual locations may also be set. These manual locations are provided by operators, producers, etc., and each manual location also has corresponding location information, which is added to the candidate locations. In some examples, manual locations may also be directly used as jump points, depending on the specific requirements. The embodiments of the present invention do not limit this.

[0112] In one alternative embodiment, such as Figure 9As shown, the multiple candidate locations are filtered to determine the corresponding multiple jump points and location description information, including: Sub-step 902: Analyze the quality of each candidate location based on the location information.

[0113] The "excitement level" is used to quantify the quality of the plot content at candidate locations, and can be determined by quantifying factors such as audience appeal and importance. Therefore, when setting the excitement level, corresponding parameters can be set, and each parameter can be weighted to determine the score. In one example, the parameters for calculating the excitement level include: content value parameter, effect value parameter, and production value parameter. The content value parameter quantifies the degree of match between the plot content corresponding to the candidate location and the program's positioning, ensuring that the selected plot information aligns with the program's core value. The effect value parameter quantifies the attractiveness of the plot content to viewers, balancing professional judgment and user experience. The production value parameter quantifies the effect of the plot content itself, including visual effects, sound effects, etc. Each parameter corresponds to a weight, and the sum of the weights of all parameters is 100% or 1. For example, the first weight of the content value parameter is 40%, the second weight of the effect value parameter is 35%, and the third weight of the production value parameter is 25%. The above are merely examples of weights, and this embodiment of the invention does not limit the scope of the calculation.

[0114] The parameter values ​​of each parameter are calculated based on the location information of the candidate location, and then weighted by the corresponding weights to obtain the brilliance of the candidate location.

[0115] Sub-step 904: Sort each candidate point in descending order of its brilliance to determine the sorting result.

[0116] Sub-step 906: Based on the sorting results, the candidate points are arranged in chronological order to form a jump point list, and multiple jump points and point description information are determined.

[0117] Sort all candidate points in descending order of their "excitement level" to determine the corresponding sorting result. Then, retrieve candidate points according to this sorting result, starting with the candidate points with the highest "excitement level," and add them to the jump point list in chronological order. When inserting, determine the time interval between the candidate point and the jump points before and after it. If the time interval is less than a first interval threshold, delete the candidate point; otherwise, add it as a jump point to the list.

[0118] When selecting candidate jump points based on their engagement level, an interval threshold can be set. This threshold is the minimum interval (dura_min_thr) between two jump points, such as 90-120 seconds. The setting of this threshold should take into account user viewing habits; excessively frequent jumps (e.g., <1.5 minutes) will disrupt the viewing flow and allow sufficient time for each engaging video segment to unfold, creating white space.

[0119] The selection rule for candidate points can be prioritized based on their level of excitement. Candidate points with higher excitement are ranked first, ensuring they occupy the jump point position first. When inserting subsequent candidate points with lower excitement, if the time interval between them and the jump point (i.e., the already determined candidate point) is less than the interval threshold, the inserted candidate point is deleted and will not be used as a jump point. Example: If the point with rank=0 is at 1680 seconds and the point with rank=1 is at 1720 seconds, since 1720-1680=40 seconds < 90 seconds, the point with rank=1 will be skipped.

[0120] Candidate points can also be filtered based on a pre-set language model. In one optional embodiment, filtering the multiple candidate points to determine the corresponding multiple jump points and point description information includes: performing a performance analysis on the point information based on the pre-set language model and outputting a ranking result of the candidate points; based on the ranking result, constructing a jump point list of candidate points in chronological order, and determining multiple jump points and point description information. Then, candidate points are obtained according to the ranking result, starting with candidate points with high performance, and added to the jump point list in chronological order. When inserting, the time interval between the candidate point and the jump points before and after it is determined. If the time interval is less than a first interval threshold, the candidate point is deleted; if it is not less than the first interval threshold, it is added to the list as a jump point.

[0121] Another parallel implementation, such as Figure 10 As shown, the analysis of the brilliance of location information based on a pre-set language model, and the output of the ranking results of candidate locations, include: Sub-step 1002: Obtain the fourth prompt word template. The fourth prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0122] The variable parameters include: a location parameter, which is used to fill in the location information for each candidate location, thereby enabling analysis, sorting, and filtering based on the quality of each candidate location. A name parameter can also be set to fill in the program name, ensuring consistency in the pre-defined language model's processing of programs.

[0123] The fourth prompt word template can be set based on the following elements: Role positioning: Continue to use the role of "professional variety show director" to ensure the professionalism and consistency of the evaluation perspective. Clear task: The model is required to rank candidate high-energy points according to importance such as brilliance, and provide the rank and ranking_reason. To make the ranking results scientific and reasonable, a weighted evaluation system is defined for the prompt words.

[0124] Here is an example of a fourth prompt word template: As a professional variety show director, you will receive a list of potential high-energy moments (i.e., candidate points) for the program "{{show_name}}". Information for each high-energy moment will include: - timestamp: a point in time - type: Jump content type - reason: The reason for determining the cause is the same as the reason for selection. Task objective: The high-energy points are ranked and estimated according to their importance as follows: 1. Content Value (40% weight) - The degree to which the core content of the program is reflected - Its role in advancing the main storyline of the program - Information content and value 2. Audience impact (35% weight) - Intensity of emotional resonance - Peak viewing experience - Memorable points and topicality 3. Create highlights (25% weight) - Visual appeal / scene appeal - Control of rhythm and atmosphere - Innovation and uniqueness ==== Return Format ==== ATTENTION: ONLY RETURN VALID JSON WITH ```json BEGINING. DO NOTINCLUDE ANY OTHER TEXT. [ { "rank": 0, "highlight": {original high-energy point information}, "ranking_reason": "The reasons for ranking based on the evaluation criteria" }, { "rank": , "highlight": {original high-energy point information}, "ranking_reason": "The reasons for ranking based on the evaluation criteria" } ] ==== Enter Information ==== <List of High-Energy Points> {{highlight_list}} This allows us to construct a fourth prompt word template, which facilitates the subsequent construction of a fourth input data for analysis and processing.

[0125] Sub-step 1004: Based on the location information and the fourth prompt word template, the fourth input information is generated by combining them.

[0126] Sub-step 1006: Input the fourth input information into a preset language model for brilliance analysis to obtain the ranking results of the candidate points.

[0127] The fourth input information is generated by combining the location information, program name, and other information with the fourth prompt word template. This fourth input information is then fed into a pre-defined language model to filter the jump locations. The model can analyze the appeal of each candidate location and rank them, outputting the ranking results. The ranking results include the jump location's ranking number, location information, and the reason for ranking.

[0128] In an example output, the large language model returns sorted data in a predefined format, such as a JSON array. The output includes: Field 1: rank, format: integer, starting from 0 (0 represents the most exciting), example: 0, 1, 2, ... Field 2: highlight, format: fully retains the highlight information from the previous step (e.g., including timestamp, type, reason), purpose: carries the original information for subsequent processing. Field 3: ranking_reason, format: natural language description, content requirement: explains the ranking basis based on the above evaluation criteria, example: This moment is the climax of the entire reasoning process; A completely connects all the evidence to deduce the truth (high content value); the reasoning process is logically rigorous and impressive (strong audience impact); combined with tense music and close-up shots, it creates a visual impact (outstanding production highlights), resulting in the highest overall score.

[0129] An example of an output result: [ { "rank": 0, "highlight": { "timestamp": 714.16, "type": "common.emotion", "Reason": "The boss announces on the stairs, 'Only X people can leave from here,' setting the ultimate rule and a life-or-death tone for the entire story, creating immense dramatic tension and a sense of oppression." }, "ranking_reason": "Content value (40%): Extremely high. ... Audience impact (35%): Extremely strong. ... Production highlights (25%): Strong." }, { "rank": 1, "highlight": { "timestamp": 527.16, "type": "specific.plot", "reason": "Faced with armed 'thugs,' D takes the initiative and orders them to 'go down and stand with them,' subverting the audience's initial understanding of his identity and constituting a key plot twist." }, "ranking_reason": "Content value (40%): Extremely high. ... Audience impact (35%): Strong. ... Production highlights (25%): Strong. ..." }, ... ] This allows the model to automatically analyze the brilliance of each candidate location and rank them, providing reasons for the ranking.

[0130] This invention enables adaptive extraction and sorting of jump points within each program segment. Within each identified program segment, a large model is used in conjunction with predicted jump content types to extract jump points, which are then automatically sorted according to their level of interest. This ensures the quality of jump points and their viewing priority, improving user satisfaction with the "skip-watch" experience.

[0131] In this embodiment of the invention, the jump points also include segmentation points and manually selected points. In some examples, segmentation points and manually selected points have higher priority than candidate points. For instance, segmentation points and manually selected points are set first in the jump point list, and then candidate points are inserted in descending order of their excitement level to determine the candidate points with suitable timing as jump points, thus obtaining a jump point list. This ensures that the density and location of the output jump points are reasonable.

[0132] In this embodiment of the invention, a title for each jump point can be generated based on its corresponding location information and plot description information. The title can be understood as a summary of the key points of that jump point. By analyzing the plot content corresponding to the location information of each jump point, and combining this with the unit description information of multiple plot units before and after it, the plot content can be understood, thus summarizing a summary of information not exceeding a predetermined word count as the title. Location description information is generated from the jump point's time point, title, etc.

[0133] In this embodiment of the invention, a title and plot summary are set for each jump point to facilitate user understanding and selection. In some optional embodiments, input data can be generated based on a preset language model and point information. For example, the point information and the fifth prompt word model are used to generate fifth input data, which is then input into the preset language model to summarize the title and output the corresponding title.

[0134] In one example, the title can be generated based on the following fifth cue word model: Your task is to summarize the plot of a specified segment in a variety show with a title and then dissolve its referential meaning.

[0135] Given the following information: 1. Dialogue segments: seg_dialog 2. Segment description: seg_clip_data 3. Complete episode summary: epis_summary 4. Guest List: charac_names ==== Task Description ==== 1. Please summarize the actual plot of the main filming scenes: - Prioritize the dialogue and behavior of characters in charac_names - Remain objective; do not speculate on plot points not explicitly mentioned in dialogue or visuals. 2. There may be dialogue from the director or an NPC: - Pay special attention to content related to issuing missions or advancing key plot points. 3. If all the messages are meaningless advertising, please enter an empty string. 4. Dialogue information is based on the content of seg_dialog, while seg_clip_data primarily focuses on behavior. 5. Please output the results in JSON format: - Ensure that the output can be correctly loaded by Python's json.loads() - If a section has no content, the corresponding field should be set to an empty string. ==== Return Format ==== ATTENTION: ONLY RETURN VALID JSON WITH ```json BEGINING. DO NOTINCLUDE ANY OTHER TEXT. json { "matched_summary": "Plot summary, highlighting key moments", "reason": "After discovering an error in person recognition, explain the basis for the correction." } ``` ==== Enter Information ==== <seg_dialog> {{seg_dialog}}< / seg_dialog> <seg_clip_data> {{seg_clip_data}}< / seg_clip_data> <epis_summary> {{epis_summary}}< / epis_summary> <charac_names> {{charac_names}}< / charac_names> This allows us to summarize the plot content of the jump point and generate a corresponding title. The time information and title of the jump point are used as the point description information.

[0136] Based on the above embodiments, this invention provides an optional embodiment of a video key point extraction method, referring to... Figure 11 As shown: Step 1102: Based on the program name and content description, determine the program type and jump duration range of the target video.

[0137] Step 1104: Based on the content summary, determine the program segments and jump content types of the target video.

[0138] The target video is segmented based on the program segments to determine multiple video clips.

[0139] Step 1106: Obtain the video description information of the target video, and use a sliding time window to detect whether the video description information contains the program segment.

[0140] The step of using a sliding time window to detect whether the video description information contains the program segment includes: obtaining a sliding time window; detecting whether the video description information within the sliding time window contains the program segment; if it does not contain the program segment, then sliding the sliding time window backward for a first duration to continue detection; if it contains the program segment, then sliding the sliding time window backward for a second duration to continue detection, wherein the second duration is greater than the first duration.

[0141] Step 1108: Divide the target video into multiple video segments based on the detected program segments.

[0142] The audio and video content of each video segment is identified to determine multiple candidate points and point information that match the type of content to be jumped to.

[0143] Step 1110: Determine plot information based on the video description information of the target video.

[0144] Step 1112: Classify the plot information. When the classification result of the plot information is a jump content type, determine the corresponding candidate points and point information.

[0145] Step 1114: Analyze the quality of each candidate location based on the location information.

[0146] Step 1116: Sort the candidate locations in descending order of their brilliance to determine the sorting result.

[0147] Step 1118: Based on the sorting results, the candidate points are arranged in chronological order to form a jump point list, and multiple jump points and point description information are determined.

[0148] Step 1120: Set jump points and point description information for the target video.

[0149] In summary, the system can perform structured analysis based on video content, determine structural patterns, and automatically segment multiple program segments. For each program segment, it identifies candidate points for exciting plot points and then filters them based on factors such as the level of excitement and time intervals. It can also automatically identify jump points and point descriptions to enable the skip function and provide users with a high-quality viewing experience.

[0150] Based on the above embodiments, this invention provides another optional embodiment of a video key point extraction method, as described above. Figure 12 As shown: Step 1202: Obtain the first prompt word template and the content description information of the target video. The prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: analysis content and output requirements.

[0151] Step 1204: Combine the content description information of the target video with the first prompt word template to generate the first input information.

[0152] Step 1206: Input the first input information into a preset language model for semantic analysis to obtain the structural pattern information of the output target video. The structural pattern information includes program type, jump duration range, program segment and jump content type.

[0153] Step 1208: Obtain the second prompt word template. The second prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0154] Step 1210: Combine the video description information, program segment, and the second prompt word template to generate the second input information.

[0155] Step 1212: Input the second input information into the preset language model for segment detection to obtain the segmentation information of the output target video. The segmentation information includes: timestamps of segmentation points, segments before and after segmentation, and the reason for segmentation.

[0156] Step 1214: Obtain the third prompt word template, which includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0157] Step 1216: Based on the video description information of each program segment, the type of jump content, and the third prompt word template, the third input information is generated by combining them.

[0158] Step 1218: Input the third input information into the preset language model for plot understanding and obtain the point information of the output candidate points. The point information includes: the timestamp of the candidate point, the type of content to which it belongs, and the reason for filtering.

[0159] Step 1220: Obtain the fourth prompt word template. The fourth prompt word template includes variable parameters and prompt items. The prompt items include at least one of the following: task content and output requirements.

[0160] Step 1222: Based on the location information and the fourth prompt word template, combine them to generate the fourth input information.

[0161] Step 1224: Input the fourth input information into the preset language model to filter the jump points and obtain the sorting result of the candidate points.

[0162] Step 1226: Based on the sorting results, the candidate points are arranged in chronological order to form a jump point list, and multiple jump points and point description information are determined.

[0163] Step 1228: Associate the multiple point description information with the target video respectively.

[0164] The automated processing method of this invention differs from manual rule definition by utilizing the content understanding capabilities of a large-scale model to achieve fully automated jump point identification. Based on the structured pattern information output by the content understanding service, it automatically generates program types and segments using a large language model, defines the jump duration range, and predicts the corresponding jump content types for each segment. Then, it automatically segments the program through segment boundary identification. Within each program segment, it uses the large-scale model combined with pre-generated jump content types to extract candidate points for exciting plot points, sorts them in descending order of their excitement level, and inserts them into the final result. This method can shorten the adaptation cycle of new programs from several weeks to several hours, achieving automatic jump point generation within 24 hours, significantly improving the coverage efficiency of the "jump-to-watch" function and the user experience.

[0165] The embodiments of the present invention implement an intelligent content navigation mechanism, which will significantly improve the user's viewing experience and stickiness, provide technical support for video platforms, and provide strong support for enhancing product competitiveness.

[0166] Based on the above embodiments, this invention also provides a video playback method based on key points, which can jump between time points of jump points during video playback after extracting point description information, providing users with a better viewing experience.

[0167] Reference Figure 13 The diagram illustrates a flowchart of an embodiment of a video playback method based on key points according to the present invention.

[0168] Step 1302: Play the target video on the playback page. The target video has location description information for the jump points.

[0169] After associating the target video with the location description information, if the server receives a playback request from the client, it can send the video data of the target video to the client, which will then parse and render the video data and play the target video on the playback page.

[0170] like Figure 14 As shown, the target video is associated with jump points and information, such as multi-layered display based on segments, plot, etc. For example, segment 1 [00:00:00-00:15:12] case introduction, the jump points under it include: [00:05:12-00:06:15] character A's self-introduction.

[0171] The target video is associated with point description information. The method for extracting the point description information is the same as described in the above embodiment and will not be repeated here.

[0172] When playing a target video, mobile phones, tablets, and other terminal devices can enable a skip playback function under a set model. In one optional embodiment of the present invention, when the terminal device is in landscape mode, at least one skip operation area is set in the playback page. The skip operation area is the operation area that triggers the skip playback jump point. The position of this area can be set according to needs, for example, it can be set on the left side, the lower right side, or the top and bottom sides of the playback page. The terminal device can also switch to landscape mode to display a prompt message to remind the viewer to trigger the skip playback at the jump point.

[0173] Step 1304: In response to the first trigger operation, determine the jump point closest to the current time point based on the point description information.

[0174] While watching a target video on the playback page, if a user wants to switch to another scene, they can perform a first trigger action. The corresponding client can receive the first trigger action, such as swiping up or down, or gesture operations. In response to the first trigger action, the current playback time of the video is determined, and the nearest jump point to the current playback time point is selected from the point description information as the target jump point.

[0175] like Figure 15 As shown, in response to the first triggering operation, such as a swipe operation, the time jump is initiated from the time point of story 1 to the time point of story 2.

[0176] The first trigger operation includes a forward jump operation and / or a backward jump operation. If a forward jump operation is received, the jump point corresponding to the previous time point of the current playback time point is searched in the point description information and used as the target jump point. For example, based on a received downward swipe operation, the jump point corresponding to the previous time point of the current playback time point is searched in the point description information and used as the target jump point, from which playback begins.

[0177] If a jump-back operation is received, the jump point corresponding to the next time point after the current playback time point is found in the point description information and used as the target jump point. For example, based on a received swipe-up operation, the jump point corresponding to the next time point after the current playback time point is found in the point description information and used as the target jump point, and playback starts from that target jump point.

[0178] Step 1306: Jump the target video from the current time point to the time point corresponding to the jump point and play it.

[0179] Obtain the video data corresponding to the target jump point, parse and render the video data, and then jump the target video to the video data rendered at the target jump point for playback.

[0180] In this embodiment of the invention, the target video can jump between various jump points. In response to a second trigger operation, playback jumps between jump points, that is, from the current jump point to the nearest target jump point. The second trigger operation is the operation that triggers the jump between jump points; the first trigger operation and the second trigger operation can be the same operation or different operations, and this embodiment of the invention does not impose any restrictions on this. Upon receiving the second trigger operation, playback jumps between jump points in response to the second trigger operation. If the second trigger operation is a forward jump operation (or playback of the previous scene), then the previous jump point of the current jump point is determined, and playback begins from that previous jump point. If the second trigger operation is a backward jump operation (or playback of the next scene), then the next jump point of the current jump point is determined, and playback begins from that next jump point. This enables video data to jump between different story segments, improving the user's video viewing experience.

[0181] This approach preserves the complete narrative of long videos while allowing users to enjoy a viewing pace similar to short videos, effectively improving user experience and platform retention.

[0182] Reference Figure 15 The diagram illustrates a flowchart of another embodiment of the video playback method based on key points according to the present invention.

[0183] Step 1502: Play the target video on the playback page. The target video has location description information for the jump points.

[0184] Step 1504: In response to the first trigger operation, determine the jump point closest to the current time point based on the point description information.

[0185] Step 1506: Jump the target video to the time point corresponding to the jump point and play it.

[0186] Step 1508: Display the title of the jump point on the playback page.

[0187] If the server receives a playback request from the client, it can send the target video data to the client. The client then parses and renders the video data and plays the target video on the playback page. The client receives a first trigger operation, such as an up / down swipe or a gesture. In response to this first trigger operation, if a forward jump operation is received, the server searches the point description information for the first jump point before the current playback time, and uses this as the target jump point. If a backward jump operation is received, the server searches the point description information for the first jump point after the current playback time, and uses this as the target jump point.

[0188] Obtain the video data corresponding to the target jump point, parse and render the video data, jump to the video data rendered from the target jump point to start playing, and display the title on the playback page.

[0189] Based on the above embodiments, this invention provides a video playback system, such as... Figure 16 As shown, the system includes: a server 1602 and a client 1604, wherein: Server 1602 determines the structural pattern information of the target video based on the content description information of the target video. The content description information is a structured description of the summary content of the target video. The structural pattern information is used to characterize the structural features of the content in the target video. Based on the structural pattern information, multiple jump points and point description information corresponding to the target video are determined, and jump points and point description information are set for the target video.

[0190] Client 1604 plays the target video on the playback page and, in response to the first trigger operation, determines the jump point closest to the current time point based on the point description information; jumps the target video from the current time point to the time point corresponding to the jump point and plays it.

[0191] The video key point extraction and playback method of this invention is of significant value to long-form video services on video platforms. It innovatively solves the problem of users encountering uninteresting content while watching long videos. When users feel that the current plot is not engaging or relatable, they no longer need to exit the video or aimlessly fast-forward. Instead, they can easily jump to the next point to continue watching by using a first trigger action, such as swiping in the left or right quarter of the screen. This "second chance" mechanism effectively reduces user abandonment rates because users are more inclined to try jumping to other exciting segments to satisfy their viewing needs than to directly exit the video.

[0192] This invention also provides an electronic device, such as... Figure 17As shown, it includes a processor 171, a communication interface 172, a memory 173, and a communication bus 174, wherein the processor 171, the communication interface 172, and the memory 173 communicate with each other through the communication bus 174. Memory 173 is used to store computer programs; When processor 171 executes a program stored in memory 173, it performs the following steps: Based on the content description information of the target video, the structural pattern information of the target video is determined; Based on the structural pattern information, multiple jump points and point description information corresponding to the target video are determined; A jump point and a jump point description are set for the target video so that when the target video is played on the playback page, in response to the first trigger operation, the jump point closest to the current time point is determined based on the jump point description information, and the video is played from the current time point.

[0193] In another alternative embodiment, when the processor 171 executes the program stored in the memory 173, it performs the following steps: Playing a target video on the playback page, the target video is associated with point description information, the point description information includes: the time point corresponding to the jump point, the jump point is obtained based on the structural pattern information of the target video, the structural pattern information is determined based on the content description information of the target video, the content description information is a structured description of the summary content of the target video, and the structural pattern information is used to characterize the structural features of the content in the target video; In response to the first trigger operation, the jump point closest to the current time point is determined based on the point description information; Jump to the target video at the time point corresponding to the jump point and play it.

[0194] The communication bus mentioned in the terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0195] The communication interface is used for communication between the aforementioned terminal and other devices.

[0196] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0197] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0198] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the video key point extraction method and the key point-based video playback method described in any of the above embodiments.

[0199] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the video key point extraction method and the key point-based video playback method described in any of the above embodiments.

[0200] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0201] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0202] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0203] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for key point extraction of a video, characterized by, The method includes: Based on the content description information of the target video, the structural pattern information of the target video is determined. The content description information is a structured description of the summary content of the target video, and the structural pattern information is used to characterize the structural features of the content in the target video. Based on the structural pattern information, multiple jump points and point description information corresponding to the target video are determined, and the point description information is used to describe the time and content of the jump points. A jump point and a jump point description are set for the target video so that when the target video is played on the playback page, in response to the first trigger operation, the jump point closest to the current time point is determined based on the jump point description information, and the video is played from the current time point to the nearest jump point.

2. The method of claim 1, wherein, The content description information includes the program name and a brief content summary, which includes the theme and a summary of the content. The determination of the structural pattern information of the target video based on the content description information of the target video includes: Based on the program name and content description, determine the program type and jump duration range of the target video; Based on the content summary, the program segments and jump content types of the target video are determined.

3. The method of claim 1, wherein, The step of parsing the content structure of the target video based on content description information to determine the structural pattern information of the target video includes: Obtain the first prompt word template, which includes variable parameters and prompt items. The variable parameters include name variables and description variables, and the prompt items include at least one of the following: analysis content and output requirements. The content description information of the target video is combined with the first prompt word template to generate the first input information; The first input information is input into a preset language model for semantic analysis to obtain the structural pattern information of the output target video. The structural pattern information includes program type, jump duration range, program segment and jump content type.

4. The method of claim 1, wherein, The step of determining multiple jump points and point description information corresponding to the target video based on the structural pattern information includes: Based on the structural pattern information, multiple candidate points in the target video are determined; The multiple candidate locations are filtered to determine the corresponding multiple jump points and location description information.

5. The method of claim 4, wherein, The program type includes the jump content type, and the determination of multiple candidate points in the target video based on the structural pattern information includes: The target video is segmented based on the program segments to determine multiple video clips; Based on the audio-visual content of each video segment, multiple candidate points and point information that match the jump content type are determined, where the audio-visual content refers to the content presented in the video segment in terms of both visuals and audio.

6. The method of claim 5, wherein, The segmentation of the target video based on the program segments to determine multiple video segments includes: The video description information of the target video is obtained, and a sliding time window is used to detect whether the video description information contains the program segment. The video description information is used to describe the plot content of the target video for a predetermined duration. The target video is divided into multiple video segments based on the detected program segments.

7. The method of claim 6, wherein, The step of detecting whether the video description information contains the program segment using a sliding time window includes: Get the sliding time window; Detect whether the video description information within the sliding time window includes the program segment; If the program segment is not included, the detection will continue for the first duration within the sliding time window; If the program segment is included, the detection will continue for a second duration within the sliding time window, where the second duration is longer than the first duration.

8. The method of claim 5, wherein, The segmentation of the target video based on the program segments includes: Obtain a second prompt word template, which includes variable parameters and prompt items. The variable parameters include: stage variables and descriptive variables. The prompt items include at least one of the following: task content and output requirements. The video description information, program segments, and the second prompt word template are combined to generate the second input information; The second input information is input into a preset language model for segment detection to obtain the segmentation information of the output target video. The segmentation information includes: timestamps of segmentation points, segments before and after segmentation, and the reason for segmentation.

9. The method of claim 5, wherein, The process of determining multiple candidate locations and location information that match the type of content to be redirected, based on the audio-visual content of each video segment, includes: Based on the video description information of the target video, determine the plot information; The plot information is categorized, and when the categorization result of the plot information is a jump content type, the corresponding candidate points and point information are determined.

10. The method of claim 5, wherein, The process of determining multiple candidate locations and location information that match the type of content to be redirected, based on the audio-visual content of each video segment, includes: Obtain a third prompt word template, which includes variable parameters and prompt items. The variable parameters include plot parameters and type parameters, and the prompt items include at least one of the following: task content and output requirements. The third input information is generated by combining the video description information, the type of content to be redirected, and the third prompt word template. The third input information is input into a preset language model for plot understanding to obtain the point information of the candidate points. The point information includes: the timestamp of the candidate point, the type of content to which it belongs, and the reason for filtering.

11. The method according to claim 4, characterized in that, The step of filtering the multiple candidate locations to determine the corresponding multiple jump points and location description information includes: The excitement level of each candidate location is analyzed based on the location information. The excitement level is used to quantitatively characterize the excitement level of the plot content of the candidate location. Sort the candidate locations in descending order of their brilliance to determine the ranking result; Based on the sorting results, the candidate points are arranged in chronological order to form a jump point list, and multiple jump points and point description information are determined.

12. The method according to claim 1, characterized in that, The step of filtering the multiple candidate locations to determine the corresponding multiple jump points and location description information includes: Based on the pre-set language model and location information, the ranking results of the candidate locations are obtained; Based on the sorting results, the candidate points are arranged in chronological order to form a jump point list, and multiple jump points and point description information are determined.

13. The method according to claim 12, characterized in that, The ranking result of candidate points based on a pre-set language model and point information includes: Obtain the fourth prompt word template, which includes variable parameters and prompt items. The variable parameters include: location parameters, and the prompt items include at least one of the following: task content and output requirements. The fourth input information is generated by combining the location information and the fourth prompt word template. The fourth input information is input into a preset language model to filter jump points and obtain the sorting results of the candidate points.

14. A video playback method based on key points, characterized in that, The method includes: Playing a target video on the playback page, the target video is set with jump point description information, the jump point description information includes: the time point corresponding to the jump point, the jump point is obtained based on the structural pattern information of the target video, the structural pattern information is determined based on the content description information of the target video, the content description information is a structured description of the summary content of the target video, and the structural pattern information is used to characterize the structural features of the content in the target video; In response to the first trigger operation, the jump point closest to the current time point is determined based on the point description information; Jump the target video from the current time point to the time point corresponding to the jump point and play it.

15. The method according to claim 14, characterized in that, The first triggering operation includes a forward jump operation and / or a backward jump operation. The response to the first trigger operation, determining the nearest jump point to the current time point based on the point description information, includes: If a forward jump operation is received, the jump point corresponding to the previous time point of the current playback time point is found in the point description information; If a jump request is received, the jump point corresponding to the next time point after the current playback time point is found in the point description information.

16. The method according to claim 14, characterized in that, Also includes: In response to the second trigger operation, jump from the current jump point to the nearest target jump point.

17. The method according to claim 14, characterized in that, Also includes: When the terminal device is in landscape mode, at least one jump operation area is set in the playback page; The first triggering operation includes a sliding operation received in the jump operation area.

18. The method according to claim 14, characterized in that, Also includes: When playback jumps to the specified time point, the title of the jump point is displayed on the playback page.

19. A video playback system, characterized in that, The system includes: a server and a client; The server determines the structural pattern information of the target video based on the content description information of the target video. The content description information is a structured description of the summary content of the target video. The structural pattern information is used to characterize the structural features of the content in the target video. Based on the structural pattern information, the server determines multiple jump points and point description information corresponding to the target video and sets jump points and point description information for the target video. The client plays the target video on the playback page and, in response to the first trigger operation, determines the jump point closest to the current time point based on the point description information; jumps the target video from the current time point to the time point corresponding to the jump point and plays it.

20. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-18.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-18.

22. A computer program product comprising a computer program / computer-executable instructions, wherein, When the computer program / computer-executable instructions are executed by a processor in an electronic device, they implement the method described in any one of claims 1-18.

Images