Video key point extracting and playing method and device, medium and program product

Abstract

The embodiment of the invention provides a video key point extraction and playing method and device, a medium and a program product, and the extraction method comprises the steps: obtaining video description information of a plurality of video clips in a target video, the video description information comprises a time point sequence of a plot unit, and each time point corresponds to the audio and video content of the plot unit; based on the audio and picture content, performing multi-dimensional plot understanding on the plot unit, screening to obtain a wonderful plot unit and determining wonderful plot information of the video clip, the wonderful plot information including a time point corresponding to the wonderful plot unit; and establishing association between the wonderful plot information of each video clip and the target video so as to jump to play the wonderful plot unit when the target video is played on the playing page. The story content can be accurately analyzed in combination with the story unit, the time point of the wonderful story is positioned, and jump playing is carried out between the wonderful story.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a method for extracting key points from a video, a method for playing a video based on key points, a video playing system, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rise of short videos, users have become accustomed to the short, fast, and precise pace of content, and are keen on the high-octane content that short video platforms continuously deliver. In contrast, traditional long videos often contain a lot of foreshadowing and transitional content due to the plot's development. This linear narrative model is significantly at odds with current user viewing habits. When users encounter clips with slow plot pacing or unengaging content, they easily lose patience and often skip uninterested content by fast-forwarding or dragging the progress bar.

[0003] However, fast-forwarding and dragging the progress bar only adjust the playback time of the video, and the adjusted content is unknown. Therefore, users need to try repeatedly to find the content they want to watch. Blindly fast-forwarding can easily miss important plots and make it impossible to accurately locate the exciting content, which in turn wastes time.

[0004] Therefore, a technical problem that those skilled in the art urgently need to solve is: how to identify the wonderful plots in the video so as to jump between the wonderful plots for playback. Summary of the Invention

[0005] One of the objectives of the embodiments of the present invention is to provide a method for extracting key points from a video to accurately identify the highlights in the video, enable jump playback between the highlights, and enhance user interest and experience. The specific technical solution is as follows:

[0006] In a first aspect of the present invention, a method for extracting key points from a video is provided, comprising: obtaining video description information of a plurality of video clips in a target video, the video description information comprising a time point sequence of a plot unit, each time point corresponding to the audio and video content of the plot unit; based on the audio and video content, screening out wonderful plot units and determining wonderful plot information of the video clips through a multi-dimensional plot understanding of the plot unit, the wonderful plot information comprising: a time point corresponding to the wonderful plot unit; associating the wonderful plot information of each video clip with the target video, so that when the target video is played on a playback page, in response to a first trigger operation, the wonderful plot unit closest to the current time point is determined based on the wonderful plot information, and the target video is jumped to the time point corresponding to the wonderful plot unit and played.

[0007] In the second aspect of the implementation of the present invention, a video playback method based on key points is also provided, including: playing a target video on a playback page, the target video is associated with wonderful plot information, the wonderful plot information includes: a time point corresponding to a wonderful plot unit, the wonderful plot unit is obtained based on a multi-dimensional plot understanding and screening of the audio and video content of the plot unit, and the plot unit is sorted in time point order in the video description information corresponding to the video clip of the target video; in response to a first trigger operation, determining the wonderful plot unit closest to the current time point based on the wonderful plot information; jumping the target video to the time point corresponding to the wonderful plot unit and playing it.

[0008] In a third aspect of the implementation of the present invention, a video playback system is also provided, comprising: a server and a client; wherein the server obtains video description information of multiple video clips in a target video, the video description information comprising a time point sequence of plot units, each time point corresponding to the audio and video content of the plot unit; based on the audio and video content, a multi-dimensional plot understanding is performed on the plot units, the plot units where the plot climax is located are screened as wonderful plot units and wonderful plot information is determined, the wonderful plot information comprising: the time point corresponding to the wonderful plot unit; the client plays the target video on the playback page, and in response to a first trigger operation, determines the wonderful plot unit closest to the current time point based on the wonderful plot information; the target video is jumped to the time point corresponding to the wonderful plot unit and played.

[0009] In another aspect of the implementation of the present invention, an electronic device is provided, characterized in that it includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement the steps of the video key point extraction method as described in the embodiment of the present invention, and the steps of the video playback method based on key points as described in the embodiment of the present invention when executing the program stored in the memory.

[0010] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned methods for extracting key points from a video and the method for playing a video based on key points.

[0011] In another aspect of the present invention, a computer program product comprising instructions is provided. When the computer program product is run on a computer, the computer executes any of the above-mentioned methods for extracting key points from a video and the method for playing a video based on key points.

[0012] The video key point extraction method provided by the embodiment of the present invention obtains the video description information of multiple video clips in the target video, and the video description information includes a time point sequence of the plot unit, each time point corresponds to the audio and video content of the plot unit, and the audio and video content describes the plot from multiple dimensions such as audio and picture, providing a data basis for the analysis of the plot. Then, based on the audio and video content, the plot unit is subjected to a multi-dimensional plot understanding, and the plot unit where the plot climax is located is screened as a wonderful plot unit and the wonderful plot information of the video clip is determined. The plot can be accurately understood in combination with the audio and video content of the plot unit, and the wonderful plot unit of the video clip can be located. Each piece of wonderful plot information is associated with the target video, so that when the target video is played on the playback page, in response to the first trigger operation, it jumps from the target video to the time point corresponding to the wonderful plot unit and plays it, and can jump and play between wonderful plots, meeting the user's viewing needs for wonderful plots and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0014] Figure 1 A flowchart of an embodiment of a method for extracting key points from a video according to the present invention;

[0015] Figure 2 This is a flowchart of the sub-steps of the step of performing multi-dimensional plot understanding on a plot unit in an embodiment of the present invention;

[0016] Figure 3 This is a flow chart of the sub-steps of extracting exciting plots in an embodiment of the present invention;

[0017] Figure 4 This is a flow chart of sub-steps of the step of determining exciting plot information in an embodiment of the present invention;

[0018] Figure 5 A flowchart of another embodiment of a method for extracting key points from a video according to the present invention;

[0019] Figure 6 A flowchart of a sub-step of extracting key plot units and generating exciting plot information in an embodiment of the present invention;

[0020] Figure 7 Flowchart of sub-steps of analysis and processing based on a preset language model in an embodiment of the present invention;

[0021] Figure 8 A schematic diagram of an example of analysis and processing based on a preset language model according to an embodiment of the present invention;

[0022] Figure 9 A flowchart of another embodiment of a method for extracting key points from a video according to the present invention;

[0023] Figure 10 A flowchart of an embodiment of a video playback method based on key points of the present invention;

[0024] Figure 11 A schematic diagram of an example of plot jump according to an embodiment of the present invention;

[0025] Figure 12 A schematic diagram of an example of a play page according to an embodiment of the present invention;

[0026] Figure 13 A flowchart of another embodiment of a video playback method based on key points of the present invention;

[0027] Figure 14 is a schematic diagram of an embodiment of a video playback system of the present invention;

[0028] Figure 15 FIG. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

[0030] An embodiment of the present invention provides a method for extracting key points of a video, which can understand the plot of the video based on multi-dimensional audio and video content, screen the wonderful plot unit where the plot climax is located, obtain wonderful plot information and establish an association with the target video, so that when the target video is played on the terminal side, in response to the first trigger operation, the wonderful plot unit closest to the current time point is determined based on the wonderful plot information, and the target video is jumped from the target video to the time point corresponding to the wonderful plot unit and played. It can jump and play between wonderful plots, meet the user's viewing needs for wonderful plots, and improve user experience.

[0031] Reference Figure 1 , shows a step flow chart of an embodiment of a method for extracting key points from a video of the present invention.

[0032] Step 102: Obtain video description information of multiple video clips in the target video.

[0033] For the target video to be analyzed, the video can be divided into multiple video clips. In one example, the video clips can be segmented according to time. For example, a 45-minute (min) video can be divided into a video clip every 4 minutes or 5 minutes. In another example, the video can also be segmented according to the plot to obtain multiple video clips. Each video clip corresponds to a plot content, so each target video can correspond to a plot list. Taking a 43-minute video as an example, it can contain 10 plots, and each plot is a relatively complete event composed of multiple scenes. Each plot is regarded as an interval, that is, divided into a video clip.

[0034] Each video clip is associated with video description information, which is used to describe the plot content of the video clip. This video description information can be obtained by identifying the video data corresponding to the video clip, including identifying and determining the video image and audio data. The video description information can also be determined based on plot information provided by the producer of the target video, although this is not limited in this embodiment of the present invention.

[0035] The description of the plot content in a video clip can be divided into multiple plot units for description. For example, the video clip can be divided into a plot unit every 15-20 seconds. Each plot unit describes the plot from multiple dimensions such as pictures and audio, thereby refining the video clip into individual plot units for description. This detailed information organization provides a data basis for subsequent analysis, which can improve the accuracy and comprehensiveness of the analysis. Each plot unit corresponds to time information and audio and video content. The time information of the plot unit can be the time range of the plot or a time point. The time point can be the starting time point of the plot or other specified time point in the plot, such as the time point containing a character. In an optional embodiment, the time point of the plot unit is a timestamp. In the video description information, the plot units can be sorted according to the time point, that is, the video description information includes a time point sequence of the plot units, each time point in the sequence corresponds to a plot unit, and each plot unit is described by audio and video content.

[0036] The audiovisual content is the visual and audio content presented in the plot unit. By recognizing each video frame and audio data in the plot unit, the visual content can be identified, such as the characters, environment, objects, and the relationships between them. The audio content can also be identified, such as the dialogue between characters and sound effects such as background music and interludes. In one example, by recognizing the video and audio data of a plot unit, the following audiovisual content is obtained: In the woods at night, two girls in ancient costumes fall from the sky. One girl is wearing a pink and white dress, and the other is wearing a blue and white dress. They first float in the air, then fall to the ground, covered in fallen leaves. Ultimately, both girls lie prone on the ground.

[0037] Step 104 , based on the audiovisual content, through multi-dimensional plot understanding of the plot unit, screen out wonderful plot units and determine wonderful plot information of the video clip, the wonderful plot information including: the time point of the plot unit.

[0038] The exciting plot unit refers to an attractive content unit in a video clip, which is usually a plot unit where the exciting clips and key plot points are located in the video clip.

[0039] The audio and visual content of each plot unit in the video description of each video clip is analyzed separately. Semantic understanding of the audio and visual content determines the plot content presented. A comprehensive evaluation of the plot content presented in each plot unit is then conducted to determine how each plot contributes to the development of the main plot. The plot unit containing the climax is then selected. The climax is the most engaging part of the plot. The plot unit containing the climax is considered a highlight unit, and highlight plot information is generated based on the time point and audio and visual content of the plot unit.

[0040] Step 106, associate the wonderful plot information of each video clip with the target video, so that when the target video is played on the playback page, in response to the first trigger operation, the wonderful plot unit closest to the current time point is determined based on the wonderful plot information, and the target video is jumped to the corresponding time point of the wonderful plot unit and played.

[0041] After extracting the highlights, the information can be associated with the target video. For example, jump points can be set in the target video according to the time points of the highlights in the information. Another example is to bind the highlights to a predetermined operation so that the video can jump between different time points based on the predetermined operation. Thus, during video playback, upon receiving a predetermined operation, the video can jump to the next time point and start playing the video data from that time point. This allows users to jump to the next highlights when they are not interested in the current plot, thus satisfying their need for watching highlights and improving the user experience.

[0042] In summary, the video description information of multiple video clips in the target video is obtained, and the video description information includes a time point sequence of the plot unit, and each time point corresponds to the audio and video content of the plot unit. The audio and video content describes the plot from multiple dimensions such as audio and picture, providing a data basis for the analysis of the plot. Then, based on the audio and video content, the plot unit is subjected to a multi-dimensional plot understanding, and the plot unit where the plot climax is located is screened as a wonderful plot unit and the wonderful plot information of the video clip is determined. The plot can be accurately understood in combination with the audio and video content of the plot unit, and the wonderful plot unit of the video clip can be located. Each piece of wonderful plot information is associated with the target video, so that when the target video is played on the playback page, in response to the first trigger operation, it jumps from the target video to the time point corresponding to the wonderful plot unit and plays it, and it can jump and play between wonderful plots, satisfying the user's viewing needs for wonderful plots and improving the user experience.

[0043] In an optional embodiment of the present invention, the video description information includes: an episode synopsis and plot information of the target video. The episode synopsis is used to describe the video clip from a global perspective, such as by genre, character, and other dimensions. The plot information is used to describe the plot content contained in the video clip. The plot information includes plot units, the time points of the plot units, and the audio and video content. In one example, the audio and video content of the plot units are presented in the plot information in chronological order.

[0044] In an optional embodiment, the series introduction includes: series type, core character information and main plot information. The series type is the type of the target video, and the series type includes at least one of the following: romance, urban workplace, ancient costume power struggle, suspense mystery, comedy, spy war, martial arts, etc., which are not limited in this embodiment of the present invention. The core character information is the core characters described in the target video, such as the male and female protagonists and the main supporting roles. The core character information corresponding to the video clip can be based on the core characters specifically contained in the video segment, and is used to determine the core characters of the video. The main plot information is the main plot content of the target video, which is used to grasp the overall story direction.

[0045] In an optional embodiment, the plot information also includes plot summary information. The plot summary information provides a brief overview of the plot content of the video clip, helping to understand the specific plot development of the video clip. In this embodiment of the present invention, to improve plot location accuracy, the time point of each plot unit can be accurately timestamped to two decimal places, such as 241.28, 248.24, etc. The audio and video content of the plot unit is described from both the picture and audio dimensions. Therefore, the audio and video content includes both picture and audio dimensions. The picture dimension includes a picture introduction and action descriptions, while the audio dimension includes dialogue and sound effects. The picture introduction describes the plot unit from the perspective of the scene, for example, by identifying each video frame in the plot unit to determine the picture content described. For example, it describes a scene such as a forest or city, including the positional relationships of several objects such as characters and animals, and the scene content such as shot switching. The action description describes the plot unit from the perspective of character actions. This action description includes the characters in the plot segment and the actions between them. If there are no characters, it can be left blank. The action description is determined by identifying each video frame in the plot unit. The dialogue content is used to describe the plot unit from the perspective of character dialogue. The dialogue content includes the characters in the plot clip and the dialogue between the characters. If there are no characters, it can be left blank. The dialogue content can be identified from the audio clip corresponding to the video clip. The sound effect content is used to describe the plot unit from the sound effect dimension. By identifying the audio data of the plot unit, the ambient sound and interludes of the plot unit can be identified, such as weather sound effects such as wind and rain, animal sound effects, interludes, and special sound effects. The sound effect content can represent the environment, atmosphere, and other content of the plot. For example, some special sound effects indicate that the current atmosphere is awkward, and some interludes indicate that the current atmosphere is sad. In addition, the plot unit that usually plays interludes is a relatively important plot point.

[0046] In one example, the video description information for a video clip of a fantasy drama is as follows:

[0047] ###Album series type: fairy tale drama.

[0048] Core character information: Female protagonist A, male protagonist B.

[0049] Main storyline: The heroine A has been seeking immortality since she was a child. After accidentally meeting the hero B, the two embarked on a journey together to unlock the ancient mythical object, the Wunian Stone. In order to gather the five thoughts of "love, hate, killing, greed, and goodness", they experienced various tests in X City, Y City, Z Mountain, M City, and N Village, and went from using each other to falling in love with each other and running towards each other.

[0050] ###Plot summary information (corresponding to the main film time is 239.16 S(344.92s): Heroine A and supporting character C are abducted by a monster. The monster intends to absorb supporting character C's spiritual energy for its own use, claiming it is a tribute to supporting character C. Heroine A desperately protects supporting character C, pleading with him to escape. In this critical moment, Heroine A kneels and prays for help from the gods. The monster, realizing it has been tricked, becomes enraged.

[0051] ###Description of plot unit (about 15-20 seconds each)

[0052] ...Omitted display

[0053] 270.92: In a forest at night, two little girls in ancient costumes ran hand in hand along a path. One girl wore a pink costume, the other a blue one. Above their heads, a large black shadow floated. Female protagonist A, wearing a pink costume, urged her companion to run faster. Then, the two girls gradually disappeared into the depths of the forest.

[0054] 288.48: In the woods at night, two girls in ancient costumes fell from the sky. One girl wore a pink and white dress, the other a blue and white one. They first floated in the air, then fell one after another onto the leaf-strewn ground. Eventually, both girls lay sprawled on the ground.

[0055] 294.36: In the woods at night, two little girls in period costumes snuggle together. One girl is wearing a pink and white costume, the other a blue and white one. They are the heroine A and the supporting character C. A man in a black fur coat, supporting character P, bares his fangs.

[0056] 301.32: In the woods, the heroine A, wearing a pink ancient costume, grabs the supporting character C, wearing a blue ancient costume, and tells him to walk faster. The supporting character C tells the heroine A to walk faster on her own, and the two argue. Finally, they both fall to the ground, and a person in dark clothes stands not far away.

[0057] 309.48: The video takes place in a cave environment with fallen leaves on the ground. Female protagonist A lies on the ground, saying "Supporting character C." She then stands up and runs with a man in a black robe and another girl in pink. The man in black grabs Supporting character C, who is wearing blue, by the neck.

[0058] 316.84: In the woods, the heroine A lies on the ground, looking frightened. She tells someone to run. The supporting character P, wearing furry clothing, stands next to the heroine A and tells her to run as well.

[0059] 323.48: The video takes place in a cave. Female protagonist A sits on the ground, blood at the corner of her mouth. She claims that someone possesses extraordinary spiritual energy and can help someone else increase their demonic power, saying it's their honor. Supporting character P, with a ferocious expression, declares that he can help increase their demonic power, his face changing. Supporting character P grabs female protagonist A's neck, and A's expression becomes painful. Then, supporting character P lets go, and A falls to the ground. Another person runs over, and female protagonist A calls out to that person, "Supporting character C." Finally, female protagonist A lies motionless on the ground.

[0060] 332.72: In a cave or similar setting, the ground is covered with fallen leaves. The video begins with a close-up of the face of the female protagonist, A. She is wearing a pink ancient costume, with a flower ornament in her hair. There are red marks at the corners of her mouth, resembling blood. Her expression is painful, and she calls out, "Supporting Character C." The camera then zooms out, revealing the female protagonist, A, kneeling on the ground. In front of her lies a figure in light blue ancient costume, motionless and apparently unconscious. In the background, a figure in dark clothing stands in the distance.

[0061] 337.08: The video begins in the woods. Female protagonist A, wearing a light blue period costume, lies on fallen leaves. A woman in a pink period costume stands beside her, her eyes closed. Then, a man in a black period costume with a fur collar on his shoulders stands in the woods. His hands are stained with blood, and he looks at them. The camera then returns to female protagonist A, who lies on the ground, held by a woman in a pink period costume, crying out for supporting character C. Finally, the camera zooms in, revealing the crying face of female protagonist A.

[0062] The video description information for a video clip can therefore describe the video clip from both a global and detailed perspective. Furthermore, the specific plot within the video clip can be divided into multiple plot units. Each plot unit can be described in detail from multiple perspectives, including visuals, sound effects, character dialogue, and actions, thus providing a data foundation for subsequent analysis.

[0063] Based on the above embodiment, in an optional embodiment, as Figure 2 As shown, step 104, based on the audiovisual content, by performing a multi-dimensional plot understanding on the plot unit, screening out the wonderful plot unit, includes the following sub-steps:

[0064] Sub-step 1042: determining a screening criterion based on the episode introduction of the target video.

[0065] The screening criteria are used to select exciting plot units. The screening criteria corresponding to the target video are determined based on the series introduction. For example, the screening criteria are formulated based on the series type, characters, and main plot.

[0066] In an optional embodiment, the episode introduction includes: episode type, core character information and main line information, and determining the screening criteria based on the episode introduction of the target video includes: determining the screening criteria template corresponding to the episode type; combining the core character information and main line information with the screening criteria template to generate screening criteria.

[0067] In embodiments of the present invention, different drama genres can correspond to different screening criteria. Therefore, a screening rule library can be constructed based on drama genre. This screening rule library can include different types of screening criteria templates, which can be updated as needed. Therefore, a screening criteria template corresponding to each drama genre can be determined. This screening criteria template is used to generate screening criteria. For example, the screening criteria template includes at least one of the following genres: romance, urban workplace, ancient costume conspiracy, suspense mystery, comedy, spy war, martial arts, etc. Core character information and main plot information are incorporated into the screening criteria template. For example, the character variable of the screening criteria template is populated with the core character, and the main plot variable is filled with main plot information. This integrates information from multiple dimensions to generate screening criteria. For example, the criteria for romance is based on love, focusing primarily on the emotional entanglements between characters. Alternatively, the criteria for urban workplace is based on plot points such as competition and growth in the workplace, focusing primarily on the growth and competition of characters in the workplace.

[0068] The embodiment of the present invention can construct a screening rule library for differentiated viewing preferences. Establish and maintain rules corresponding to the characteristics of exciting plots of different video types (such as action movies, suspense dramas, variety shows, etc.) to accurately identify different types of exciting plot nodes. For example, different screening standard templates are constructed based on different drama types, and a screening rule library is generated. The screening criteria can be adjusted based on demand and expanded to the rule library, for example, to support more types of video content (such as sports events, documentaries, etc.), develop identification strategies for corresponding exciting plot units, etc.

[0069] Sub-step 1044, determining the plot screening principle.

[0070] Sub-step 1046 , performing a multi-dimensional plot understanding on the audio and video content of each plot unit, and screening out the wonderful plot units according to the screening criteria and plot screening principles.

[0071] Plot screening principles are used to select compelling plot units. These principles are formulated from both a global and personalized perspective, and therefore include at least one of the following: integrity, independence, appeal, pacing, and viewing experience. Of these, integrity, independence, and viewing experience can be considered overall plot screening principles, while appeal and pacing can be considered personalized. The integrity principle maintains the integrity of the plot; the content before and after each plot unit must be relevant to avoid confusion for viewers. The independence principle maintains the independence between plot units; the content before and after each unit must remain relatively independent to facilitate understanding. If the plot remains unchanged before and after a jump, users may choose to skip to the next unit, impacting the user experience. The appeal principle selects compelling plot content for viewers. Appealing plot content can be determined for different series. For example, romances are more appealing to viewers for emotionally charged segments, while suspenseful dramas are more appealing for suspenseful plots, twists, and other plot twists. The pacing principle controls the pacing of the selected plots, and it's important to prioritize exciting plot units at the climax, rather than midway. The viewing experience principle is used to maintain the watchability of the plots, ensuring coherence and smoothness between the selected plot units, providing users with a better viewing experience. Using the above plot screening principles to select exciting plot units, it is possible to switch between relatively independent plot segments while maintaining the integrity and coherence of the plot, controlling the plot's rhythm and attracting the viewer's attention. In this embodiment of the present invention, the screening rule library also includes plot screening principles to guide the algorithm to more accurately capture key plot units in the video.

[0072] The audiovisual content of each plot unit can be used for multi-dimensional plot understanding, and the plot content presented in the plot unit can be understood through this multi-dimensional audiovisual content as the plot understanding result. The plot understanding results are then filtered according to screening criteria and plot screening principles, such as plot screening principles related to the series type and plot screening principles that control the overall situation. The plot unit containing the climax of the video clip is determined as the highlight plot unit and the highlight plot information is extracted.

[0073] In another optional embodiment, screening criteria and plot screening principles can be used to assist in multi-dimensional plot understanding of audio-visual content. For example, based on the main characters in the screening criteria, plot units containing the main characters can be screened out, and based on the screening principles, it can be determined whether the plot content of the plot unit meets the screening principles, etc. The screening criteria and plot screening principles are integrated into the multi-dimensional plot understanding of the audio-visual content to screen out wonderful plot units.

[0074] In an optional embodiment, if Figure 3As shown, the sub-step 1046 performs a multi-dimensional plot understanding on the audio and video content of each plot unit, and selects the wonderful plot units according to the screening criteria and plot screening principles, including:

[0075] Sub-step 302 : Perform multi-dimensional plot understanding on the audio and video content of each plot unit to obtain a plot understanding result of each plot unit.

[0076] The audiovisual content of each plot unit is analyzed in multiple dimensions, including semantic understanding of the content presented in multiple dimensions, such as audio and visuals, to determine the plot content presented by the plot unit. This is the plot understanding result, which represents the plot content presented by the plot unit. For example, the plot understanding result may indicate that the plot unit is a scene with intense conflict or a calm and heartwarming story. Another example is that the plot understanding result indicates that the plot unit contains physical conflict between characters or rapid scene changes.

[0077] Sub-step 304 , performing plot analysis on the plot summary information according to the screening criteria to determine key plot points of the segment.

[0078] The plot summary information is analyzed based on the screening criteria. This analysis can be combined with the screening criteria for the series type, characters, and main plot. The plot summary information can be used to determine the plot section containing the main characters, as well as the content within that plot section that is relevant to the series type and main plot, to determine the key plot points of the clip. The key plot points describe the key plot points of the video clip, such as the protagonist being kidnapped or the supporting character being captured.

[0079] Sub-step 306 , based on the plot screening principle, the key plot of the clip and the plot understanding result, the plot units are screened to determine the exciting plot units.

[0080] Based on the plot screening principles and the key plots of the clips, the plot understanding results are screened. For example, based on the key plots of the clips, several plot units corresponding to the key plots are determined, and the plot units that meet the principles are screened in combination with the plot screening principles to obtain wonderful plot units.

[0081] In an optional embodiment of the present invention, determining the exciting plot information in step 104 includes the following sub-steps:

[0082] Sub-step 402: generating a title for the wonderful plot unit based on the audio and video content of the wonderful plot unit.

[0083] Based on the audiovisual content and time point corresponding to the wonderful plot unit, the wonderful plot information is generated. Among them, the wonderful plot unit can be summarized based on the audiovisual content of the wonderful plot unit to generate a title, for example, summarizing a plot content of no more than 10-20 words as the title.

[0084] Sub-step 404: generating wonderful plot information using the title and time point of the wonderful plot unit.

[0085] The title and time point of the exciting plot unit are used as the exciting plot information.

[0086] The reason for selecting the wonderful plot unit can also be determined, for example, by combining the key plot of the clip and the screening principles, etc., to determine the reason for selecting the plot unit as the wonderful plot unit, and add the reason to the wonderful plot information.

[0087] Based on the above embodiment, the embodiment of the present invention provides a method for extracting key points of a video, such as Figure 5 As shown:

[0088] Step 502: Obtain video description information of multiple video clips in the target video.

[0089] Step 504: Determine a screening standard template corresponding to the drama type.

[0090] Step 506: Combine the core character information and the main plot information with the screening criteria template to generate screening criteria.

[0091] Step 508: Determine the plot screening principle.

[0092] Step 510 : Perform multi-dimensional plot understanding on the audio and video content of each plot unit to obtain a plot understanding result of each plot unit.

[0093] Step 512: Analyze the plot summary information according to the screening criteria to determine the key plot points of the segment.

[0094] Step 514 , based on the plot screening principle, the key plot of the clip and the plot understanding result, the plot units are screened to determine the exciting plot units.

[0095] Step 516: Generate a title for the wonderful plot unit based on the audio and video content of the wonderful plot unit.

[0096] Step 518: Generate wonderful plot information using the title and time point of the wonderful plot unit.

[0097] Step 520: Associate each piece of wonderful plot information with the target video.

[0098] Jump points are set in the target video according to the time points of the wonderful plot units in the wonderful plot information, so as to jump and play between the jump points in response to the first trigger operation.

[0099] In summary, multimodal fusion technology is used to analyze the video description information of video clips. That is, the video plot is analyzed by combining the overall story line of the video, character information, current plot summary and the video's own description of the picture, audio, etc. Compared with single modality analysis, it can more comprehensively understand the video content, improve the accuracy of identifying exciting plots, and reduce the omission of important plots.

[0100] In an optional embodiment, if Figure 6 As shown, the step 514 is to screen the plot units based on the plot screening principle, the key plot of the clip and the plot understanding result to determine the wonderful plot units, including:

[0101] Sub-step 602 , based on the multi-dimensional plot screening principles and the key plots of the clips, the plot units are screened according to the plot understanding results, and at least one plot unit corresponding to each plot screening principle is determined.

[0102] The plot screening principles include plot screening principles of the global dimension and plot screening principles of the series type dimension. Among them, based on the principle of integrity and the key plot of the segment, the audio and visual content of the plot unit is understood to determine at least one plot unit; based on the principle of independence and the key plot of the segment, the audio and visual content of the plot unit is understood to determine at least one plot unit; based on the principle of attractiveness and the key plot of the segment, the audio and visual content of the plot unit is understood to determine at least one plot unit; based on the principle of rhythm and the key plot of the segment, the audio and visual content of the plot unit is understood to determine at least one plot unit; based on the principle of viewing experience and the key plot of the segment, the audio and visual content of the plot unit is understood to determine at least one plot unit. For each plot screening principle, the plot understanding results can be combined with the key plot of the segment to screen the plot understanding results, analyze whether the plot understanding results of the plot unit are the key plot of the segment, and whether they meet the corresponding plot screening principles. For example, for the principle of integrity and the principle of independence, the plot understanding results of the previously screened wonderful plot units can be matched to determine whether the plot remains complete and relatively independent. For the viewing experience principle, the image frame corresponding to the timestamp can be combined to determine whether the picture is plot content, etc., to avoid the picture falling into an empty space and other positions, which affects the user's viewing experience.

[0103] Sub-step 604 , comparing at least one plot unit corresponding to each plot screening principle, and determining the plot unit that meets the most plot screening principles as the wonderful plot unit.

[0104] The plot units selected by each plot screening principle can be intersected to determine the plot units that meet all the plot screening principles. If the intersection is empty, the plot unit that meets the most plot screening principles is determined. If there is more than one plot unit that meets the most plot screening principles, such as more than one plot unit in the intersection, each plot unit selected by the plot screening principle can be assigned a weight. Based on this weight information, a weighted calculation can be performed to select the plot unit with the highest weight as the highlight plot unit.

[0105] In this way, the plot units contained in the fragments can be understood through a variety of plot screening principles and key plots of the fragments, and the plot units can be screened through a variety of principles, so that the screened plot units can highlight the key plots on the basis of ensuring the smoothness and completeness of the plot, thereby improving the user experience.

[0106] In another optional embodiment, the step 510, based on the plot screening principles and key plot points of the clips, performs a multi-dimensional plot understanding of the audio and video content of each plot unit, and screens the plot unit where the plot climax is located, including: performing a multi-dimensional plot understanding of the picture introduction, dialogue content, sound effect content and action description of the plot unit, and obtaining the plot understanding result of the plot unit.

[0107] The multi-dimensional contents described in the plot unit, such as the pictures, character exchanges, character actions, and sound effects, are analyzed. Among them, semantic understanding can be performed by combining the multi-dimensional information such as the picture introduction, dialogue content, sound effect content, and action description of the plot unit to determine the plot content presented in the plot unit as the plot understanding result of the plot unit.

[0108] In some other optional embodiments, the plot understanding of the plot unit can also be performed in combination with the plot screening principles and / or the key plot of the segment. The corresponding plot understanding results can record whether it is the focus of the segment, whether it meets certain screening principles, etc., such as whether the plot content is complete and relatively independent, whether the communication and actions between the characters are attractive, whether it is the climax of the plot, etc., so that the plot unit where the plot climax is located can be determined as a wonderful plot unit.

[0109] Therefore, it is possible to analyze the plot units contained in the clips through the plot screening principles and the key plots of the clips, analyze the plots of the units through multiple dimensions such as the pictures, character communication, actions, sound effects, etc., and screen the plot units that meet the needs. The screened plot units can highlight the key plots and improve the user experience while ensuring the smoothness and completeness of the plot.

[0110] Based on the above embodiment, the plot understanding analysis of the video description information can also be performed using a language model. By training a preset language model to understand the semantic content of the video description information and perform analysis, the efficiency of the analysis can be improved. Therefore, in another optional embodiment, step 104, based on the audio and video content, by performing a multi-dimensional plot understanding on the plot unit, screening out the wonderful plot unit and determining the wonderful plot information of the video clip, includes the following sub-steps: performing a multi-dimensional plot understanding on the video description information based on a preset language model, and outputting the wonderful plot information corresponding to the video clip.

[0111] The embodiment of the present invention pre-trains a language model to extract exciting plot information, wherein the language model is an abstract mathematical model constructed based on objective facts of language and is a basic tool in the field of computational linguistics. Natural language processing (NLP) is performed based on the language model to understand the semantics of the video description information, thereby extracting exciting plot information from the video clip. The preset language model can be various language models based on NLP, such as a natural language processing model based on the Transformer model, a multi-task learning model, a neural network model, and a large language model (LLM). Among them, the LLM model is a deep learning model trained using a large amount of text data, so that the model can generate natural language text or understand the meaning of language text.

[0112] Input information is constructed based on the video description information, and the input information is input into a preset language model for plot understanding processing, thereby outputting the highlight plot information of the plot unit where the plot climax of the video clip occurs. The highlight plot information includes: the title and time point of the highlight plot unit. In one example, the highlight plot information also includes: the reason for selecting the highlight plot unit.

[0113] In an embodiment of the present invention, the preset language model can also be optimized in real time, and feedback information can be determined in combination with the user's operations during playback, or a feedback interface can be provided to the user to obtain the user's feedback information. Based on the feedback information, the preset language model and prompt word template are optimized to improve the accuracy of model processing.

[0114] Based on the above embodiments, Figure 7 As shown, the multi-dimensional plot understanding of the audio and video content based on the preset language model and the output of the wonderful plot information corresponding to the video clip include the following sub-steps:

[0115] Sub-step 702: obtaining a prompt word template.

[0116] Pre-set language models can be combined with prompt engineering to perform analysis and processing. Prompt engineering can be used to design and optimize input information, guiding a pre-set language model, such as an LLM model, to produce the desired output. Prompt templates can be designed based on prompts, and these templates can be used to guide a pre-set language model, such as an LLM model.

[0117] The prompt word template includes variable parameters and prompt items. Variable parameters are variables that need to be filled in the prompt word template. The variable settings are related to the video description information. The variable parameters are associated with the prompt items, and the analysis and processing operations for the video description information are determined in conjunction with the prompt items. For example, the genre variable is used to fill in the episode genre, and the character variable is used to fill in the key character information. The prompt items are prompts that guide the preset language model in executing the processing.

[0118] The prompt items include at least one of the following: analysis content, screening criteria, plot screening principles and output requirements. Among them, the analysis content is used to guide the analysis and processing operations required to be performed by the preset language model. The analysis content can also describe the basic data provided, such as providing video description information, etc. The screening criteria are used to guide the preset language model to perform the screening criteria required for analysis and processing. The plot screening principles are used to guide the preset language model to perform the plot screening principles required for analysis and processing. The setting of the plot screening principles can be similar to the above. The plot screening principles include at least one of the following: integrity principle, independence principle, attractiveness principle, rhythm principle and viewing experience principle. For details, please refer to the description of the embodiment part above, which will not be repeated here. The output requirements are used to determine the content and format of the output results of the preset language model, such as requiring the model to input the time point, title and screening reason of the screened exciting plot unit.

[0119] An example of a prompt word template is as follows, where the double curly braces {{}} are required to be filled with variable parameters:

[0120] ========Analysis Content=========

[0121] You are an experienced video editor who needs to find the best moments in a video to capture the audience's attention. I will provide: 1. Video segment description data (focused on screen description); 2. Plot summary of the current segment; 3. Main storyline; 4. Information about key characters.

[0122] =========Screening Criteria=========

[0123] 1. Related to the specified main characters: {{charac_names}}

[0124] 2. The plot is exciting and can attract the audience. Different themes have different focuses, so you need to judge the audience's preferences based on the main storyline. Here are some examples:

[0125] Romance dramas: Plot descriptions related to the emotional development and changes of the male and female protagonists are given a higher weight, because the core of romance dramas is love, and the audience is more concerned about the emotional entanglements between the protagonists and the final outcome.

[0126] Urban workplace dramas focus on depicting contemporary workplace life and the protagonist's career growth and transformation within the workplace. Therefore, script evaluations prioritize the realistic nature of the plot, the characters' professionalism and developmental journeys, the brutality and complexity of workplace competition, and the reflection and exploration of social realities. Emotional plotlines typically serve as a secondary theme, showcasing the characters' emotional lives and interpersonal relationships, and should not overshadow the main plot.

[0127] Ancient costume dramas center on the struggle for power and the manipulation of public opinion. Therefore, script evaluations prioritize the logical and rigorous nature of the plot, the ingenuity and cunning of the characters, the intensity of the political struggles, the fidelity of the historical setting (if historical context is used), and the depth and purpose of the themes. While emotional plots may be present, they typically serve the main plot and do not dominate.

[0128] Suspense and mystery dramas focus on puzzle solving and suspense. Therefore, script evaluation focuses on the plot's suspenseful setting, the logical and rigorous reasoning, the cleverness of the twists and turns, the control of pacing, and the convincingness of the final revelation. Character development is also important, but the emphasis is placed on the complexity and multifaceted nature of the characters, rather than simply emotional portrayal.

[0129] Comedy: The core of a comedy is to create laughter and entertain the audience. Therefore, script evaluation focuses on whether the jokes are dense and effective, whether the comedy plots are cleverly designed, whether the characters' comedic performances are effective, and the overall atmosphere. The plot logic can be relatively loose, but it must maintain a certain degree of plausibility.

[0130] =========Plot Screening Principles=========

[0131] 1. The context is complete: jumping to that point will not confuse the audience

[0132] 2. Information independence: The content before and after this point is relatively independent, making it easier to understand.

[0133] 3. Appeal: Capture the audience’s attention immediately

[0134] 4. Rhythm control: Avoid choosing the middle point of the plot climax

[0135] 5. Viewing experience: The content after the jump should be coherent and smooth

[0136] ========Output requirements=========

[0137] 1. Each split point requires: an accurate time point, a summary of the core content, and an explanation of why it is suitable as a jump point

[0138] 2. Point selection principle: choose the starting point of the plot / scene, avoid choosing the middle point of the climax, and ensure that the audience can quickly understand the current situation after jumping

[0139] =======Return format========

[0140]

[0141]

[0142] =======Input information========

[0143] Plot summary: {{input_desc}}, Main storyline: {{main_storyline}}, Video clips: {{clips_data}}

[0144] In one optional embodiment, the aforementioned prompt template can be used to set a universal prompt template for various drama types, and the type variable can be set to correspond to the screening criteria for the drama type. The prompt template is designed based on the purpose and needs of video analysis, and includes guidance on identifying characteristics of different themes. For example, for fairy tale themes, special attention will be paid to elements such as emotional ties and the challenges faced by the protagonist, while for suspense themes, more emphasis will be placed on clues and reversal moments. In another optional embodiment, a prompt template can be set for each drama type, and the prompt template sets the screening criteria corresponding to the type, so that the prompt template for the corresponding drama type can be directly obtained.

[0145] The embodiment of the present invention can guide the processing of the preset language model by setting a prompt word template. The setting of the prompt word template can be flexibly set based on the analysis requirements, thereby guiding the preset language model to perform accurate analysis and processing, and improving the accuracy of the processing results.

[0146] Sub-step 704: combining the video description information and the prompt word template to generate input information.

[0147] The video description information is entered into the variable parameters of the prompt word template according to the data content to generate the input information for the preset language model. For the video description information of the video clip, the plot units are organized in chronological order and combined with the episode summary, and then input into the large model through the preset prompt template.

[0148] Sub-step 706: input the input information into a preset language model for multi-dimensional plot understanding, and output exciting plot information corresponding to the video clip.

[0149] The input information constructed by the prompt word template and the video description information is input into the preset language model for analysis. The preset language model can combine the guidance of the prompt word to perform the corresponding understanding of the plot content and the analysis and screening of the wonderful plot, thereby extracting the wonderful plot unit and determining the title and time point of the wonderful plot unit. It can also generate the screening reason for extracting the wonderful plot unit, etc., and generate the wonderful plot information as the output result.

[0150] The pre-built language model comprehensively considers multiple dimensions during the analysis process. For example, it assesses the completeness of the scene, ensuring that the selected time points fully convey a plot unit. For example, in the example above, the conflict scene at 323.48 seconds fully depicts the demon's true identity. It also determines the criticality of the plot, focusing on key turning points that drive the story forward. Furthermore, it considers the audience's viewing experience, ensuring that the selected points quickly capture attention and facilitate understanding of the plot's direction. Upon completion of the analysis, the large model outputs results that include the precise time points, an overview of key content, and detailed justification for the selected points.

[0151] In the example above video clip, the output of the model is:

[0152] ###Large model returns

[0153] 'time_code':323.48,

[0154] 'key_moment':'Supporting character P grabs the female protagonist A's neck, claiming to absorb her spiritual energy to enhance her demonic power',

[0155] 'reason':'This is a crucial turning point and a flashpoint for conflict: 1) It reveals the true purpose of supporting character P in kidnapping the sisters; 2) Female protagonist A's life is in danger, bringing the plot to a climax; 3) Jumping here provides complete context (the previous capture and the current encounter), allowing the audience to quickly understand the situation; 4) This scene kicks off the entire crisis scenario, rather than cutting in mid-sentence; 5) It showcases the crucial character of Female protagonist A, who is willing to sacrifice herself to protect supporting character C.'}

[0156] On the basis of the above embodiment, in order to maintain the continuity between the plots, the intervals between the wonderful plots should not be too long. Therefore, the time intervals between the plot units can also be detected, and a recursive strategy can be used to screen the wonderful plot units. For example, first identify the wonderful plot units from the overall plot level, and construct an initial time point sequence based on the time points of the wonderful plot units. For the intervals between two adjacent time points in the initial time point sequence with too large a time interval, the system will further perform a secondary identification of the wonderful plot units in the interval, thereby obtaining a more fine-grained time point. This recursive process will continue until the time intervals between all adjacent time points meet the reasonable interval requirements. This coarse-to-fine, layer-by-layer processing method not only ensures that important plot nodes are not missed, but also maintains an appropriate sense of rhythm, so that the short video finally generated contains the essence of the plot and has a good viewing experience.

[0157] In an optional embodiment, the time points of the wonderful plot units extracted from each video clip are arranged in chronological order to determine the time interval between two adjacent time points; for two target time points whose time interval exceeds a set time threshold, the video description information of the video clip between the two target time points is determined to perform multi-dimensional plot understanding and determine the wonderful plot information based on the video description information.

[0158] Each video clip corresponds to extracting a wonderful plot unit. After completing the screening of the wonderful plot units of multiple video clips, they can be sorted according to the chronological order of the time points to generate a time point sequence of the wonderful plot units. Determine the time interval between two adjacent time points in the time point sequence. Among them, the starting time point and the ending time point of the target video can also be added to the time point sequence to more accurately determine the time interval between adjacent time points. The starting time point can be the time point of the timestamp at zero o'clock, or the time point of the timestamp when the beginning of the film is skipped. The ending time point can be the time point of the timestamp at the end of the video, or the time point of the timestamp before the beginning of the film.

[0159] The time interval between any two adjacent time points is then compared with a set time threshold. If the time interval does not exceed the set time threshold, the judgment continues for the next time interval. If the time interval exceeds the set time threshold, for the two target time points whose time interval exceeds the set time threshold, the video description information of the video segment between the two target time points is determined. The two target time points are two adjacent time points whose time interval exceeds the set time threshold. For the video description information of the video segment between the two target time points, the multi-dimensional plot understanding of the video description information in the above embodiment is performed to determine the exciting plot information.

[0160] In an embodiment of the present invention, after analyzing the video description information of the first video clip and obtaining the initial time point sequence, the recursive optimization phase begins. The time intervals between adjacent time points can be calculated, and all intervals exceeding a set time threshold can be marked as regions to be optimized. For example, the time threshold can be set to 90s, 100s, 120s, 150s, etc. For each region to be optimized, the system treats it as a new video clip and re-extracts the video description information of the video clip for analysis. The algorithm continuously monitors the time intervals between adjacent time points and performs iterative analysis until the time intervals between adjacent time points fall within a reasonable time range, such as a time range of 40 to 90 seconds.

[0161] Based on the above embodiment, after recursively screening the highlight plot units, post-processing can be performed, such as testing the analyzed highlight plot information, including empty space detection and / or interval detection. This dual detection mechanism effectively improves the quality and usability of the system output results.

[0162] In an optional embodiment, the step of detecting an empty shot includes: performing image detection on the image frame corresponding to the time point of the wonderful plot unit; if it is detected that the image frame is an empty shot, adjusting the time point of the corresponding wonderful plot unit to locate the time point of the plot content in the wonderful plot unit.

[0163] Obtain the image frame corresponding to the time point of each wonderful plot unit, perform image detection on the image frame, and determine whether the picture content of the image frame is an empty shot, wherein an empty shot refers to a shot outside the plot content. Usually, an empty shot is a cutscene for plot transition or scene switching. Although it may contain beautiful scene shots, it often lacks substantial plot content. Jumping based on this time point will affect the watchability of the plot. If it is detected that the picture of the image frame is an empty shot, adjust the time point of the corresponding wonderful plot unit. For example, the timestamp can be adjusted forward or backward to adjust the time point when the picture of the image frame contains the plot content as the time point of the wonderful plot unit, so as to locate the time point of the plot content in the wonderful plot unit. Empty shots are usually also misjudgment points of model recognition. Therefore, empty shot detection can improve the accuracy of plot positioning and improve user experience.

[0164] In an optional embodiment, interval detection can also be performed, wherein, in the above-mentioned iterative analysis process, there may be a video clip to be processed between adjacent time points whose time interval exceeds the set time threshold, and it is always impossible to extract a suitable wonderful plot unit. In this case, the video clip to be processed can be marked, and after the overall analysis is performed, interval detection is performed again, and other decisions are introduced to locate the time point of the jump. Transition point data can be introduced as an auxiliary decision-making basis to determine the corresponding time point of the appropriate transition and the wonderful plot information corresponding to the time point. This processing not only ensures that the distribution of the time points of the wonderful plot on the time axis is relatively uniform, avoids the situation where the time points are too dense or there is an excessively long blank period, but also ensures that the generated time point sequence maintains the integrity of the content.

[0165] Based on the above embodiment, the embodiment of the present invention further provides a method for extracting key points of a video, such as Figure 8 、 Figure 9 As shown:

[0166] Step 902: Obtain video description information of multiple video clips in the target video.

[0167] Step 904: Obtain a prompt word template.

[0168] Step 906: Combine the video description information and the prompt word template to generate input information.

[0169] Step 908: Input the input information into a preset language model for multi-dimensional plot understanding, and output exciting plot information corresponding to the video clip.

[0170] After combining the video description information and the prompt word template to generate input information, it is input into the preset language model, and analyzed based on the plot tension, visual impact, plot coherence, etc., to output exciting plot information.

[0171] In step 910, the time points of the wonderful plot units extracted from each video clip are arranged in chronological order, and the time interval between two adjacent time points is determined.

[0172] Step 912: Determine whether the time interval exceeds a set time threshold.

[0173] If yes, that is, the time interval exceeds the set time threshold, step 914 is executed; if no, that is, the time interval does not exceed the set time threshold, the process returns to step 910 and continues to determine the time interval between the next adjacent time points.

[0174] Time interval detection is used to detect the intervals between exciting plot units and refine the granularity of detection and analysis.

[0175] Step 914: Determine the video description information of the video segment between the two target time points.

[0176] The two target time points refer to two adjacent time points whose time interval exceeds a set time threshold. Then, the analysis and processing process of steps 904 to 908 is iterated until the time interval between any two adjacent time points does not exceed the time threshold.

[0177] The embodiment of the present invention has a dynamic duration constraint mechanism, which avoids over-crowding of exciting points by setting a reasonable plot time interval (such as 40-180 seconds), thereby ensuring the continuity of the user's viewing experience.

[0178] After obtaining the wonderful plot information, post-processing can be performed to improve the extraction accuracy of the wonderful plot unit.

[0179] Step 916: perform image detection on the image frames corresponding to the time points of the wonderful plot units.

[0180] Step 918: Determine whether the image frame is a blank shot.

[0181] If so, it is detected that the image frame is empty, and step 920 is executed; if not, it is detected that the image frame is not empty, and the process returns to step 916 to continue detecting until all image frames at each time point are retrieved.

[0182] Step 920: Adjust the time point of the corresponding wonderful plot unit to locate the time point of the plot content in the wonderful plot unit.

[0183] Step 922: Set a jump point in the target video according to the time point of the wonderful plot unit in the wonderful plot information.

[0184] The embodiment of the present invention uses the content understanding ability of the preset language model combined with the plot rules of expert experience to accurately analyze and locate long videos, thereby achieving intelligent conversion into a form similar to short videos.

[0185] Targeting the principle of content understanding. In order to make the preset language model's understanding of the plot closer to the real viewing experience of human audiences, the embodiment of the present invention processes the video picture information to achieve this. Unlike simple script text analysis, when watching a video, the audience will pay more attention to the specific details in the picture, such as the behavior of the characters, the scene environment, the interaction between characters and other visual information. The embodiment of the present invention integrates existing visual analysis services to obtain structured information including scene descriptions, character behaviors, interactive relationships, etc., and then uses the preset language model to integrate and understand multi-dimensional information. These local information are correlated with the overall story line to evaluate the role of each plot in promoting the development of the main plot. This way of understanding combined with the main line enables the system to distinguish between key plots and minor branches, and will not miss fragments that are crucial to understanding the overall story, thereby more accurately simulating the viewing experience and judgment logic of human audiences.

[0186] Modeling plot patterns. The embodiment of the present invention establishes a rule library for identifying exciting scenes based on different plot types to guide the large model to more accurately capture key moments in the video. For example, in a suspense drama, the rule library will emphasize close-up shots, changes in character expressions, or key dialogues when clues are revealed; in idol dramas, more attention is paid to elements such as interactive details and atmosphere creation when emotional relationships turn. Through this rule-guided approach, the preset language model can more accurately identify the exciting clips that can most resonate with a specific audience of the subject matter, thereby providing a better viewing experience.

[0187] Targeted at segmentation optimization strategy. The embodiment of the present invention adopts a recursive strategy for identifying exciting plot units. First, the most exciting moments of the main plot nodes are identified at the overall plot level, and an initial sequence of time points for exciting plot units is constructed. For intervals with too large intervals in these initial time point sequences (exceeding the set time threshold), the system will further perform a secondary identification of exciting plot units in the intervals, thereby obtaining more fine-grained time points. This recursive process will continue until the time intervals of all adjacent points meet the reasonable time interval requirements. This coarse-to-fine, layer-by-layer processing method not only ensures that important plot nodes are not missed, but also maintains an appropriate sense of rhythm, so that the exciting plot information finally generated contains both the essence of the plot and a good viewing experience.

[0188] These various principles work together in concert. Content understanding provides the foundation for accurate recognition, plot pattern modeling ensures that recognition results align with user expectations, and segment optimization strategies, through recursive processing, ensure a balanced distribution of compelling clips. Leveraging this multi-layered theoretical foundation, embodiments of the present invention can transform traditional long-form video content into shorter, more responsive formats that align with contemporary viewing habits, while maintaining the integrity and appeal of the original plot.

[0189] In terms of operations, the above-mentioned extraction technology can assist operators in quickly screening and editing video materials, improve work efficiency, and provide material support for short video secondary creation. In an optional embodiment, the video data of the target video can also be edited based on the wonderful plot information to generate a video of a set length. The wonderful plot information can provide material support for editing. Based on the wonderful plot information, the plot in the target video can be screened and re-edited to obtain a video of a set length, such as a behind-the-scenes video, a plot introduction video, and other secondary creation videos.

[0190] In an optional embodiment, a delivery point may be determined based on the exciting plot information, and delivery information may be added to the delivery point. Based on the exciting plot information, the plot type of the plot in the target video, such as a climax, a smooth part, a transition part, etc., is determined, and the corresponding delivery point is set. Delivery information, such as an advertisement, is set and played at the delivery point, thereby ensuring advertising effectiveness without affecting the user experience.

[0191] The embodiments of the present invention implement an intelligent content navigation mechanism, which will significantly improve the user's viewing experience and stickiness, provide technical support for the video platform, and provide strong support for improving product competitiveness.

[0192] Based on the above embodiments, an embodiment of the present invention also provides a video playback method based on key points, which can jump between the time points of the wonderful plot units during the video playback after extracting the wonderful plot information, providing users with a better viewing experience.

[0193] Reference Figure 10 , shows a step flow chart of an embodiment of a video playback method based on key points of the present invention.

[0194] Step 1002: Play a target video on a play page, wherein the target video is associated with exciting plot information.

[0195] After associating the target video with the exciting plot information, if the server subsequently receives a playback request from the client, it can send the video data of the target video to the client, parse and render the video data on the client, and play the target video on the playback page.

[0196] like Figure 11 As shown, the target video is associated with exciting plot information, for example, the exciting plot unit of plot 1 is positioned at 22 minutes and 30 seconds into the video, and is titled B scolds A; the exciting plot unit of plot 2 is positioned at 23 minutes and 54 seconds into the video, and is titled A discovers the spirit stone.

[0197] The target video is associated with exciting plot information. The method for extracting the exciting plot information is described in the above embodiment and will not be described in detail here.

[0198] When playing a target video, a mobile phone, tablet computer, or other terminal device can enable the jump play function under a set model. In an optional embodiment of the present invention, when the terminal device is in landscape mode, at least one jump operation area is set in the playback page. The jump operation area is an operation area for triggering the jump play of a highlight plot unit. The location of this area can be set based on needs, for example, on the left side, lower right side, or both sides of the playback page. The terminal device can also be switched to landscape mode to display a prompt message to prompt the viewing user to trigger the jump play of the highlight plot unit.

[0199] In one example, operation prompt information can be displayed on the playback page, such as Figure 12 In the example shown, there are jump control areas on the left and right sides of the playback page, with a prompt like "Scan up and down to jump to different episodes." The left area has a brightness control area with a prompt like "Swipe up and down to adjust brightness," and the right area has a volume control area with a prompt like "Swipe up and down to adjust volume." While watching a video, users can perform operations in the corresponding areas to adjust brightness, volume, and jump between different episodes.

[0200] Step 1004 : In response to the first trigger operation, determine the exciting plot unit closest to the current time point based on the exciting plot information.

[0201] While watching a target video on the playback page, if a user wishes to switch to another segment, the user may perform a first trigger operation, and the corresponding client may receive the first trigger operation, such as an up / down swipe operation, a gesture operation, etc. In response to the first trigger operation, the current playback time of the video is determined, and the highlight unit closest to the current playback time is determined in the highlight information as the target highlight unit.

[0202] like Figure 11 As shown, in response to a first trigger operation such as a sliding operation, the time point of plot 1 jumps to the time point of plot 2.

[0203] The first trigger operation includes a forward jump operation and / or a backward jump operation. If a forward jump operation is received, the highlight plot information is searched for a highlight plot unit corresponding to a time point before the currently playing time point, and this is used as the target highlight plot unit. For example, based on a received downward swipe operation, the highlight plot information is searched for a highlight plot unit corresponding to a time point before the currently playing time point, and this is used as the target highlight plot unit, and playback begins from this target highlight plot unit.

[0204] If a backward jump operation is received, the highlight plot unit corresponding to the time point after the current playback time point is searched in the highlight plot information and used as the target highlight plot unit. For example, based on a received upward swipe operation, the highlight plot unit corresponding to the time point after the current playback time point is searched in the highlight plot information and used as the target highlight plot unit, and playback starts from the target highlight plot unit.

[0205] Step 1006: Jump the target video to the time point corresponding to the exciting plot unit and play it.

[0206] The video data corresponding to the target wonderful plot unit is obtained, the video data is parsed and rendered, and the target video is jumped to the video data rendered by the target wonderful plot unit for playback.

[0207] In an embodiment of the present invention, the target video can jump between various wonderful plot units. In response to a second trigger operation, the target video jumps and plays between the wonderful plot units. The second trigger operation is an operation that triggers the jump between the wonderful plot units. The first trigger operation and the second trigger operation can be the same operation or different operations. The embodiment of the present invention does not limit this. After receiving the second trigger operation, the target video jumps and plays between the wonderful plot units in response to the second trigger operation. If the second trigger operation is a forward jump operation (or called playing the previous plot), the previous wonderful plot unit of the current wonderful plot unit is determined, and the video jumps to start playing from the previous wonderful plot unit. If the second trigger operation is a backward jump operation (or called playing the next plot), the next wonderful plot unit of the current wonderful plot unit is determined, and the video jumps to start playing from the next wonderful plot unit. This enables the video data to jump between various plot units, improving the user's video viewing experience.

[0208] The above method not only retains the complete narrative of long videos, but also allows users to enjoy a viewing rhythm similar to short videos, effectively improving user experience and platform retention rate.

[0209] Reference Figure 13 , shows a step flow chart of another embodiment of a video playback method based on key points of the present invention.

[0210] Step 1302: Play the target video on the play page, where the target video is associated with exciting plot information.

[0211] Step 1304 : In response to the first trigger operation, determine the exciting plot unit closest to the current time point based on the exciting plot information.

[0212] Step 1306: Jump the target video to the time point corresponding to the exciting plot unit and play it.

[0213] Step 1308: Display the title of the wonderful plot unit on the play page.

[0214] If the server receives a play request from the client, it can send the video data of the target video to the client, which then parses and renders the video data and plays the target video on the playback page. The client receives a first trigger operation, such as an up / down swipe operation or a gesture operation. In response to the first trigger operation, if a forward jump operation is received, the server searches the highlight information for the first highlight unit before the current playback time point, and uses this as the target highlight unit. If a backward jump operation is received, the server searches the highlight information for the first highlight unit after the current playback time point, and uses this as the target highlight unit.

[0215] Obtain the video data corresponding to the target wonderful plot unit, parse and render the video data, jump the target video to the video data rendered from the target wonderful plot unit to start playing, and display the title on the play page. Figure 11 As shown, in response to the first trigger operation, the play starts from the time point of plot 1 to the time point of plot 2, and the title of plot 2 is displayed on the play page: A discovers the spirit stone.

[0216] Currently, some users, accustomed to watching short videos, are easily distracted when watching long videos. They often choose to exit directly when encountering uninteresting clips, ultimately leading to a continuous decline in user retention rates on long video platforms. Short videos refer to videos whose total length does not exceed the first time threshold, such as a total length of no more than 3, 4, 5, or 6 minutes, while long videos refer to videos whose total length exceeds the second time threshold, such as a total length of more than 20, 25, 30, 40, or 45 minutes.

[0217] The embodiment of the present invention can solve the problem of the separation between the long video viewing experience and user habits. By introducing the advantages and characteristics of short videos into the long video viewing scene, the intelligent regulation of the content rhythm is realized. While maintaining the complete narrative of the long video, it meets the viewing users' demand for rapid consumption of climax content. Specifically, the present invention uses the above-mentioned processing steps to perform structured analysis on long videos, conducts in-depth understanding and evaluation of the video content, sets screening rules and standards in combination with different plot types (such as suspense, ancient costume idol dramas, fairy tales, modern idol dramas, etc.), realizes intelligent identification and extraction of wonderful plot units, and adopts a multi-stage processing method to refine the plot of the video clips and reorganize the long video into attractive viewing units. Through simple and intuitive sliding gestures, sliding and other triggering operations, users can jump freely between these wonderful clips and enjoy a viewing rhythm similar to that of a short drama. This innovative processing method not only retains the complete narrative of the long video, but also allows users to enjoy a viewing rhythm similar to that of a short video, effectively improving the user experience and platform retention rate.

[0218] Based on the above embodiments, the embodiment of the present invention provides a video playback system, such as Figure 14 As shown, the system includes: a server 1402 and a client 1404, wherein:

[0219] Server 1402 obtains video description information of multiple video clips in the target video, where the video description information includes a time point sequence of plot units, and each time point corresponds to the audio and video content of the plot unit; based on the audio and video content, through multi-dimensional plot understanding of the plot units, screens out wonderful plot units and determines wonderful plot information, where the wonderful plot information includes: the time points corresponding to the wonderful plot units; and associates each piece of wonderful plot information with the target video.

[0220] Client 1404 plays the target video on the playback page, and in response to the first trigger operation, determines the exciting plot unit closest to the current time point based on the exciting plot information; jumps the target video to the time point corresponding to the exciting plot unit and plays it.

[0221] The method for extracting and playing key points of a video according to an embodiment of the present invention is of great value to the long video business of a video platform. It innovatively solves the problem of users encountering uninteresting content when watching long videos. That is, when users feel that the current plot cannot resonate or is not attractive enough, they no longer need to exit the video or fast-forward aimlessly. Instead, they can use the first trigger operation, such as easily sliding on the left and right 1 / 4 areas of the screen, to quickly jump to the next exciting plot unit and continue watching. This "second chance" mechanism can effectively reduce the user's abandonment rate, because compared to directly exiting the video, users are more inclined to try to jump to other exciting clips to meet their needs for chasing dramas.

[0222] The embodiment of the present invention further provides an electronic device, such as Figure 15 As shown, it includes a processor 151, a communication interface 152, a memory 153 and a communication bus 154, wherein the processor 151, the communication interface 152, and the memory 153 communicate with each other through the communication bus 154.

[0223] Memory 153, for storing computer programs;

[0224] The processor 151 is configured to execute the program stored in the memory 153 to implement the following steps:

[0225] Obtain video description information of multiple video clips in a target video, the video description information including a time point sequence of plot units, each time point corresponding to the audio and video content of the plot unit; based on the audio and video content, through a multi-dimensional plot understanding of the plot unit, screen out wonderful plot units and determine the wonderful plot information of the video clips, the wonderful plot information including: the time point corresponding to the wonderful plot unit; establish an association between the wonderful plot information of each video clip and the target video, so that when the target video is played on the playback page, in response to a first trigger operation, the wonderful plot unit closest to the current time point is determined based on the wonderful plot information, and jumps from the target video to the time point corresponding to the wonderful plot unit and plays it.

[0226] Among them, the video description information also includes: an episode introduction of the target video, and the screening of wonderful plot units based on the audio and video content through multi-dimensional plot understanding of the plot units includes: determining screening criteria based on the episode introduction of the target video; determining plot screening principles, conducting multi-dimensional plot understanding of the audio and video content of each plot unit, and screening out wonderful plot units according to the screening criteria and plot screening principles.

[0227] Among them, the drama introduction includes: drama type, core character information and main line information. The screening criteria are determined based on the drama introduction of the target video, including: determining the screening criteria template corresponding to the drama type; combining the core character information and main line information with the screening criteria template to generate screening criteria.

[0228] Among them, the video description information also includes: plot summary information, the multi-dimensional plot understanding of the audio and video content of each plot unit, and screening to obtain wonderful plot units according to the screening criteria and plot screening principles, including: multi-dimensional plot understanding of the audio and video content of each plot unit to obtain the plot understanding results of each plot unit; plot analysis of the plot summary information according to the screening criteria to determine the key plot of the clip; screening the plot units based on the plot screening principles, the key plot of the clip and the plot understanding results to determine the wonderful plot units.

[0229] Among them, the screening of plot units based on the plot screening principles, key plots of the clips and plot understanding results to determine the wonderful plot units includes: based on multi-dimensional plot screening principles and key plots of the clips, screening the plot units according to the plot understanding results, and determining at least one plot unit corresponding to each plot screening principle, the plot screening principles include plot screening principles of the global dimension and plot screening principles of the series type dimension; comparing at least one plot unit corresponding to each plot screening principle, and determining the plot unit that meets the most plot screening principles as the wonderful plot unit.

[0230] Among them, the audio and video content includes picture dimension content and audio dimension content, the picture dimension content includes picture introduction and action description, and the audio dimension content includes dialogue content and sound effect content; the multi-dimensional plot understanding of the audio and video content of each plot unit to obtain the plot understanding results of each plot unit includes: multi-dimensional content plot understanding of the picture introduction, dialogue content, sound effect content and action description of the plot unit to obtain the plot understanding results of the plot unit.

[0231] The determining of the wonderful plot information includes: generating a title of the wonderful plot unit based on the audio and video content of the wonderful plot unit; and generating the wonderful plot information using the title and time point of the wonderful plot unit.

[0232] Among them, based on the audio and video content, through multi-dimensional plot understanding of the plot unit, wonderful plot units are screened out and wonderful plot information of the video clip is determined, including: multi-dimensional plot understanding of the video description information based on a preset language model, and outputting the wonderful plot information corresponding to the video clip.

[0233] Among them, the multi-dimensional plot understanding of the audio and video content based on the preset language model and the output of the wonderful plot information corresponding to the video clip include: obtaining a prompt word template, the prompt word template includes variable parameters and prompt items, and the prompt items include at least one of the following: analysis content, screening criteria, plot screening principles and output requirements; combining the video description information and the prompt word template to generate input information; inputting the input information into the preset language model for multi-dimensional plot understanding, and outputting the wonderful plot information corresponding to the video clip, the wonderful plot information also including: the title of the wonderful plot unit.

[0234] The method further includes: performing image detection on the image frame corresponding to the time point of the wonderful plot unit; if the image frame is detected to be blank, adjusting the time point of the corresponding wonderful plot unit to locate the time point of the plot content in the wonderful plot unit.

[0235] It also includes: arranging the time points of the wonderful plot units extracted from each video clip in chronological order to determine the time interval between two adjacent time points; for two target time points whose time interval exceeds a set time threshold, determining the video description information of the video clip between the two target time points, so as to perform multi-dimensional plot understanding and determine the wonderful plot information based on the video description information.

[0236] The associating each wonderful plot information with the target video includes: setting jump points in the target video according to the time points of the wonderful plot units, so as to jump and play between the jump points in response to the first trigger operation.

[0237] The method further includes: editing the video data of the target video based on the wonderful plot information to generate a video of a set length.

[0238] In another optional embodiment, the processor 151 is configured to implement the following steps when executing the program stored in the memory 153:

[0239] A target video is played on a playback page, wherein the target video is associated with exciting plot information, wherein the exciting plot unit is obtained based on a multi-dimensional plot understanding and screening of the audio and video content of the plot unit, and the plot unit is sorted in time order in the video description information corresponding to the video clip of the target video; in response to a first trigger operation, the exciting plot unit closest to the current time point is determined based on the exciting plot information; the target video is jumped to the time point corresponding to the exciting plot unit and played.

[0240] Among them, the first trigger operation includes a forward jump operation and / or a backward jump operation, and the response to the first trigger operation is to determine the wonderful plot unit closest to the current time point based on the wonderful plot information, including: if a forward jump operation is received, then the wonderful plot unit corresponding to the previous time point of the currently played time point is searched in the wonderful plot information; if a backward jump operation is received, then the wonderful plot unit corresponding to the next time point of the currently played time point is searched in the wonderful plot information.

[0241] It also includes: in response to the second trigger operation, jumping and playing between the wonderful plot units.

[0242] Among them, it also includes: when the terminal device is in landscape mode, setting at least one jump operation area in the playback page; the first trigger operation includes a sliding operation received in the jump operation area.

[0243] The method further includes: when jumping to the time point for playback, displaying the title of the wonderful plot unit on the playback page.

[0244] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0245] The communication interface is used for communication between the above terminal and other devices.

[0246] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0247] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0248] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the video key point extraction method and the key point-based video playback method described in any of the above embodiments.

[0249] In another embodiment provided by the present invention, a computer program product containing instructions is also provided. When the computer is run on the computer, the computer executes the video key point extraction method and the key point-based video playback method described in any of the above embodiments.

[0250] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0251] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0252] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are described briefly because they are generally similar to the method embodiments. For related portions, reference can be made to the description of the method embodiments.

[0253] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for extracting key points from a video, characterized in that: The method comprises: Obtaining video description information of multiple video clips in a target video, wherein the video description information includes a time point sequence of a plot unit, each time point corresponding to audio and video content of the plot unit; Based on the audiovisual content, through multi-dimensional plot understanding of the plot unit, a wonderful plot unit is screened out and wonderful plot information of the video clip is determined, wherein the wonderful plot information includes: a time point corresponding to the wonderful plot unit; The wonderful plot information of each video clip is associated with the target video, so that when the target video is played on the playback page, in response to the first trigger operation, the wonderful plot unit closest to the current time point is determined based on the wonderful plot information, and the target video is jumped to the corresponding time point of the wonderful plot unit and played.

2. The method according to claim 1, characterized in that The video description information also includes: an episode introduction of the target video, wherein based on the audio and video content, a multi-dimensional plot understanding of the plot unit is performed to screen out the wonderful plot unit, including: determining screening criteria based on the episode synopsis of the target video; Determine the plot screening principles; Conduct a multi-dimensional understanding of the audiovisual content of each plot unit, and select the wonderful plot units according to the screening criteria and plot screening principles.

3. The method according to claim 2, characterized in that The episode introduction includes: episode type, core character information and main plot information. The screening criteria determined based on the episode introduction of the target video include: Determine a screening standard template corresponding to the drama type; The core character information and the main line information are combined with the screening criteria template to generate screening criteria.

4. The method according to claim 2, characterized in that The video description information also includes: plot summary information, The multi-dimensional plot understanding of the audiovisual content of each plot unit is carried out, and the wonderful plot units are screened according to the screening criteria and plot screening principles, including: Conduct multi-dimensional plot understanding of the audio and video content of each plot unit to obtain the plot understanding results of each plot unit; Perform plot analysis on the plot summary information according to the screening criteria to determine the key plot points of the clip; Based on the plot screening principles, key plot points of the clips and plot understanding results, the plot units are screened to determine the exciting plot units.

5. The method according to claim 4, characterized in that The plot units are screened based on the plot screening principles, key plots of the clips and plot understanding results to determine the wonderful plot units, including: Based on the multi-dimensional plot screening principles and the key plots of the segments, the plot units are screened according to the plot understanding results, and at least one plot unit corresponding to each plot screening principle is determined. The plot screening principles include the plot screening principles of the global dimension and the plot screening principles of the series type dimension; Compare at least one plot unit corresponding to each plot screening principle, and determine the plot unit that meets the most plot screening principles as the wonderful plot unit.

6. The method according to claim 4, characterized in that The audiovisual content includes picture dimension content and audio dimension content, the picture dimension content includes picture introduction and action description, and the audio dimension content includes dialogue content and sound effect content; The multi-dimensional plot understanding of the audio and video content of each plot unit is performed to obtain the plot understanding results of each plot unit, including: Perform multi-dimensional plot understanding on the screen introduction, dialogue content, sound effect content and action description of the plot unit to obtain a plot understanding result of the plot unit.

7. The method according to any one of claims 1 to 6, characterized in that: The determining of the exciting plot information includes: Generating a title for the wonderful plot unit based on the audio and video content of the wonderful plot unit; Use the title and time point of the exciting plot unit to generate exciting plot information.

8. The method according to claim 1, characterized in that The method of screening out wonderful plot units and determining wonderful plot information of the video clip based on the audiovisual content and performing multi-dimensional plot understanding on the plot units includes: The video description information is subjected to multi-dimensional plot understanding based on a preset language model, and the exciting plot information corresponding to the video clip is output.

9. The method according to claim 8, characterized in that The multi-dimensional plot understanding of the audio and video content based on the preset language model and outputting the wonderful plot information corresponding to the video clip includes: Obtaining a prompt word template, the prompt word template including variable parameters and prompt items, the prompt item including at least one of the following: analysis content, screening criteria, plot screening principles, and output requirements; Combining the video description information and the prompt word template to generate input information; The input information is input into a preset language model for multi-dimensional plot understanding, and the wonderful plot information corresponding to the video clip is output, where the wonderful plot information also includes: a title of a wonderful plot unit.

10. The method according to claim 1, characterized in that Also includes: Performing image detection on image frames corresponding to time points of the exciting plot units; If it is detected that the image frame is an empty shot, the time point of the corresponding wonderful plot unit is adjusted to locate the time point of the plot content in the wonderful plot unit.

11. The method according to claim 1, wherein Also includes: Arrange the time points of the exciting plot units extracted from each video clip in chronological order and determine the time interval between two adjacent time points; For two target time points whose time interval exceeds a set time threshold, video description information of the video segment between the two target time points is determined, so as to perform multi-dimensional plot understanding and determine exciting plot information based on the video description information.

12. The method according to claim 1, characterized in that The step of associating each piece of wonderful plot information with the target video includes: Jump points are set in the target video according to the time points of the wonderful plot units, so as to jump and play between the jump points in response to the first trigger operation.

13. The method according to claim 1, wherein Also includes: The video data of the target video is edited based on the wonderful plot information to generate a video of a set length.

14. A video playback method based on key points, characterized in that: The method comprises: Playing a target video on a play page, the target video being associated with exciting plot information, the exciting plot information including: time points corresponding to exciting plot units, the exciting plot units being obtained based on a multi-dimensional plot understanding and screening of the audio and visual content of the plot units, the plot units being sorted in time order in the video description information corresponding to the video clips of the target video; In response to a first trigger operation, determining a wonderful plot unit closest to the current time point based on the wonderful plot information; Jump the target video to the corresponding time point of the exciting plot unit and play it.

15. The method according to claim 14, characterized in that The first trigger operation includes a forward jump operation and / or a backward jump operation, In response to the first trigger operation, determining the exciting plot unit closest to the current time point based on the exciting plot information includes: If a forward jump operation is received, the wonderful plot unit corresponding to the time point before the current playing time point is searched in the wonderful plot information; If a backward jump operation is received, the wonderful plot unit corresponding to the next time point after the current playing time point is searched in the wonderful plot information.

16. The method according to claim 14, characterized in that Also includes: In response to the second trigger operation, the playback jumps between the exciting plot units.

17. The method according to claim 14, characterized in that Also includes: When the terminal device is in landscape mode, at least one jump operation area is set in the play page; The first trigger operation includes a sliding operation received in the jump operation area.

18. The method according to claim 14, characterized in that Also includes: When jumping to the time point for playback, the title of the wonderful plot unit is displayed on the playback page.

19. A video playback system, characterized in that: The system includes: a server and a client; The server obtains video description information of multiple video clips in a target video, the video description information including a time point sequence of plot units, each time point corresponding to the audio and video content of the plot unit; based on the audio and video content, performs a multi-dimensional plot understanding of the plot units, selects plot units with plot climaxes as exciting plot units, and determines exciting plot information, the exciting plot information including the time points corresponding to the exciting plot units; and associates each piece of exciting plot information with the target video; The client plays the target video on the playback page, and in response to the first trigger operation, determines the exciting plot unit closest to the current time point based on the exciting plot information; jumps the target video to the time point corresponding to the exciting plot unit and plays it.

20. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 18 when executing a program stored in a memory.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 18 is implemented.

22. A computer program product comprising a computer program / computer executable instructions, wherein: When the computer program / computer executable instructions are executed by a processor in an electronic device, the method according to any one of claims 1 to 18 is implemented.

Images