Video story visualizing method, device, equipment and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for visualizing video plots, relating to the field of video playback. The method includes: acquiring an original TV series video and multiple secondary-creation videos, wherein the secondary-creation videos are videos edited from the original TV series video; selecting target secondary-creation videos containing key plot segments of the original TV series video from the multiple secondary-creation videos; and generating a plot storyline of the original TV series video based on the target secondary-creation videos, wherein the plot storyline includes at least two of the target secondary-creation videos. This application is capable of generating plot storylines for original TV series videos based on a massive amount of secondary-creation videos.

Description

Technical Field

[0001] This application relates to the field of video playback, and in particular to a method, apparatus, device, and storage medium for visualizing video plots. Background Technology

[0002] Intellectual Property (IP) content, such as movies and TV series, is currently the most popular type of multimedia content. Major online video platforms offer exclusive IP series to attract viewers.

[0003] Since many drama enthusiasts have limited time to watch series, online video platforms provide plot summary interfaces for IP dramas. Taking a 20-episode drama as an example, this plot summary interface includes a tiered plot summary for each episode, or for every 3-5 episodes. These tiered plot summaries are compiled by staff or enthusiastic netizens.

[0004] However, the above plot summary is rather simplistic, offering only limited plot information, and the accuracy of this information is largely dependent on the expertise of staff or enthusiastic netizens. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for visualizing video storylines, capable of generating storylines from original TV series videos based on a vast amount of secondary creation videos. The technical solution is as follows:

[0006] According to one aspect of this application, a method for visualizing video plots is provided, the method comprising:

[0007] The original TV series video and multiple secondary creation videos are obtained, wherein the secondary creation videos are videos edited from the original TV series video;

[0008] From the multiple derivative videos, select target derivative videos that contain key plot segments of the original TV series videos;

[0009] The original TV series video is generated based on the target secondary creation video, and the plot storyline includes at least two of the target secondary creation videos.

[0010] According to one aspect of this application, a method for visualizing video plots is provided, the method comprising:

[0011] A storyline page that displays the first plot storyline of the original episode video, the storyline page including introductory information for at least two key plot segments belonging to the first plot storyline;

[0012] A trigger operation is performed to receive introductory information for the first of the at least two key plot segments;

[0013] In response to the triggering operation, a secondary creation video corresponding to the first key plot segment is played. The secondary creation video is a video edited from the original episode video.

[0014] According to one aspect of this application, a visualization device for video storylines is provided, the device comprising:

[0015] The video acquisition module is used to acquire original episode videos and multiple secondary creation videos, wherein the secondary creation videos are videos edited from the original episode videos.

[0016] The plot mining module is used to filter out target secondary creation videos containing key plot segments of the original TV series videos from the multiple secondary creation videos;

[0017] The plot development module is used to generate a plot storyline of the original drama video based on the target secondary creation video, and the plot storyline includes at least two of the target secondary creation videos.

[0018] According to one aspect of this application, a visualization device for video storylines is provided, the device comprising:

[0019] The display module is used to display the storyline page of the first plot storyline of the original TV series video. The storyline page includes introductory information of at least two key plot segments belonging to the first plot storyline.

[0020] The human-computer interaction module is used to receive a trigger operation for the introduction information of the first key plot segment in the at least two key plot segments;

[0021] The display module is used to respond to the trigger operation and play a secondary creation video corresponding to the first key plot segment, wherein the secondary creation video is a video edited from the original episode video.

[0022] According to one aspect of this application, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing a computer program, the computer program being executed by the processor to cause the computer device to implement the video plot visualization method as described above.

[0023] According to another aspect of this application, a computer-readable storage medium is provided, the storage medium storing a computer program that is executed by a processor to implement the video plot visualization method described above.

[0024] According to another aspect of this application, a computer program product is provided, the computer program product storing a computer program that is executed by a processor to implement the video plot visualization method as described above.

[0025] The beneficial effects of the technical solutions provided in this application include at least the following:

[0026] In the case of multiple secondary creation videos (short videos) derived from original TV series video clips, a target secondary creation video belonging to key plot segments is screened out from the multiple secondary creation videos. The plot storyline of the original TV series video is generated based on the target secondary creation video. This provides a technical solution for displaying the video plot based on secondary creation videos as the main visual element. Since the presentation of short videos is richer and more intuitive, it can provide a much more diverse way of displaying the plot than simple text and image storylines.

[0027] Furthermore, since different derivative videos may originate from different authors' editing of exciting plot segments, they can more accurately express the plot storyline that appeals to the general public, avoiding the plot storyline being limited by the thinking of individual staff members or enthusiastic netizens. Attached Figure Description

[0028] Figure 1 This invention illustrates a structural block diagram of a computer system provided in one embodiment of the present application;

[0029] Figure 2 A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0030] Figure 3 This illustration shows a schematic diagram of the construction process of a storyline provided in one embodiment of this application;

[0031] Figure 4 A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0032] Figure 5 A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0033] Figure 6 A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0034] Figure 7 This illustration shows a statistical diagram of a plot key index calculated based on "consumption signals" and "production signals" according to an embodiment of this application;

[0035] Figure 8A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0036] Figure 9 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0037] Figure 10 A flowchart illustrating a video story visualization method provided in one embodiment of this application is shown;

[0038] Figure 11 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0039] Figure 12 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0040] Figure 13 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0041] Figure 14 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0042] Figure 15 This application shows a diagram of the relevant interface for a storyline provided in one embodiment.

[0043] Figure 16 A schematic diagram illustrating the construction process of a story graph provided in one embodiment of this application is shown;

[0044] Figure 17 A block diagram of a video plot visualization apparatus provided in one embodiment of this application is shown;

[0045] Figure 18 A block diagram of a video plot visualization apparatus provided in one embodiment of this application is shown;

[0046] Figure 19 A block diagram of a computer device provided in one embodiment of this application is shown. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0048] Short video creation is currently a primary focus for self-media creators. After a popular IP drama series airs, numerous self-media creators will produce secondary creations based on that IP drama series, resulting in a massive number of short videos edited from that IP drama series. This application provides a visualization technology solution for video storylines. It generates the original drama series video's storyline based on multiple short videos, with each storyline including one or more short videos, and the storyline is presented through multiple short videos (high-energy plot clips).

[0049] Figure 1 A structural block diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system 100 includes a terminal 120 and a server 140.

[0050] Terminal 120 has an application (also called a client) installed and running. This application can be any of the following: online video application, short video application, microblogging application, browser application, instant messaging application, e-commerce application, social networking application, etc. For illustrative purposes, terminal 120 is the terminal used by the first user. Optionally, terminal 120 is logged into a user account. Terminal 120 uses the service provided by server 140 (such as online video service, short video service, search service, encyclopedia service, social networking service) through the user account. Terminal 120 includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, etc.

[0051] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0052] Server 140 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. For example, server 140 includes a processor 144 and a memory 142. Memory 142 further includes a receiving module 1421, a control module 1422, and a sending module 1423. The receiving module 1421 receives requests sent by clients, such as requests to play IP dramas, or requests to search for IP dramas, or requests to search for plot elements or characters in IP dramas, etc. The data processing module 1422 provides functions such as storyline analysis, processing, and output for IP dramas. Server 140 provides background services to clients, responds to various requests from terminal 120, and provides visualized plot information for IP dramas. Optionally, server 140 undertakes the primary computing work, and terminal 120 undertakes secondary computing work; or, server 140 undertakes secondary computing work, and terminal 120 undertakes primary computing work; or, server 140 and terminal 120 use a distributed computing architecture for collaborative computing.

[0053] This application embodiment uses a smartphone as an example to illustrate terminal 140. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds, or even more terminals. This application embodiment does not limit the number of terminals or the type of device.

[0054] Figure 2 A flowchart illustrating a video story visualization method provided in an exemplary embodiment of this application is shown. This embodiment illustrates the application of this method in a server 140. The method includes:

[0055] Step 202: Obtain the original episode videos and multiple secondary creation videos. The secondary creation videos are videos edited from the original episode videos.

[0056] Original episode videos can be in the form of movies, TV series, documentaries, animations, etc. Original episode videos are relatively long, for example, exceeding a first threshold. This first threshold is 10 minutes, 30 minutes, etc. Original episode videos can be referred to as "long videos." The production company of the original episode video can be a film or television production company.

[0057] Derivative videos are videos created by editing original episode clips. For example, one or more video segments from an original episode can be edited into a single short video. The creators of derivative videos can be independent media creators, pre-production promotional organizations for the series, or enthusiastic netizens. The creators of derivative videos may be different from those of the original episode clips.

[0058] In some embodiments, the original episode videos and the derivative videos are on the same network platform, and the server of the network platform obtains the original episode videos and multiple derivative videos from the database of the network platform. In other embodiments, the original episode videos and all or part of the derivative videos are on different network platforms, and the server obtains the original episode videos and multiple derivative videos through web crawling tools.

[0059] Exemplary Reference Figure 3 For a popular original TV series video 1, a large number of derivative videos may emerge based on it. These derivative videos could be edited by self-media author A, video account author B, or netizens C. Each derivative video includes one or more plot segments.

[0060] Step 204: Select target secondary creation videos from multiple secondary creation videos that contain key plot segments of the original drama series videos;

[0061] Because derivative works videos are often edited manually, they tend to include exciting, high-energy, sweet, and special effects shots, thus containing crucial plot segments. The server selects target derivative works videos from multiple videos that contain key plot segments from the original series video. These target derivative works videos can be one or more.

[0062] The original TV series video has first video data, which includes data related to the consumption process of the original TV series video. Illustratively, the consumption process-related data includes at least one of the following: the number of bullet comments, comments, likes, favorites, and skipped views at a given playback time. For example, at a time t, there are 26 bullet comments, 12 comments, and 3 likes; or at another time, there are 123 bullet comments, 26 comments, and 4 likes.

[0063] Secondary creation videos possess secondary video data, which includes data related to the production and / or consumption process of the secondary creation videos. Illustratively, data related to the production process includes at least one of the following: author level, number of followers, number of videos produced, and number of online platforms participated in. Data related to the consumption process includes at least one of the following corresponding to the playback time: number of bullet comments, number of likes, number of favorites, and number of views redirected to the next playback point.

[0064] In some embodiments, the server calculates the plot key index for each playback time point based on the first video data and / or the second video data. Based on the plot key indices of multiple playback time points, it filters out target secondary creation videos that belong to the plot key segments of the original drama video. Here, a playback time point refers to a time point on the playback progress bar of the original drama video. Since the secondary creation video is edited from the original drama video, aligning the video frames of the secondary creation video with those of the original drama video allows us to obtain the playback time points of each video frame in the secondary creation video.

[0065] In some embodiments, the server calculates the plot key index for each playback time point based on first video data, such as calculating the plot key index based on the number of bullet comments at each playback time point. In some embodiments, the server calculates the plot key index for each playback time point based on second video data, such as calculating the plot key index based on the number of derivative videos at each playback time point. In some embodiments, the server calculates the plot key index for each playback time point based on both first and second video data, such as by weighted summing the number of bullet comments on the original episode video at a certain playback time point and the number of derivative videos corresponding to that playback time point.

[0066] illustrative reference Figure 3 The server calculates the plot key index at multiple playback time points, resulting in Plot Key Index Curve 2. Playback time segments corresponding to one or more peak plot key indices on this curve are identified as plot key segments. For example, time point t is a local peak plot index; the plot key index at time point t is calculated as: number of bullet comments * a + number of short videos * b. The number of bullet comments is the number of bullet comments in the original video episode corresponding to time point t, or in other words, the number of bullet comments that will be played at time point t. The number of short videos is the number of short videos in the playback interval that includes time point t, or in other words, the number of short videos containing the video frame corresponding to time point t. Here, a and b are preset weights.

[0067] Since the playback range may include multiple short videos containing key plot segments, one target short video can be selected from these multiple short videos as the target secondary creation video corresponding to that key plot segment.

[0068] Step 206: Generate the plot storyline of the original drama video based on the target secondary creation video. The plot storyline includes at least two target secondary creation videos.

[0069] Based on the selected target derivative videos, the server generates one or more storylines from the original drama video, with each storyline including at least two target derivative videos.

[0070] When multiple derivative videos correspond to the same key plot point, the server filters out the derivative videos that correspond to that key plot point. After grouping the different key plot points according to the plot's structure, the derivative videos belonging to the same group are generated as a single storyline for the original episode video.

[0071] For an original TV series video, there may be one storyline or multiple storylines. In some embodiments, storylines can be obtained according to different characters in the original TV series video, such as the storyline corresponding to character A and the storyline corresponding to character B; in other embodiments, storylines can be obtained according to different events in the original TV series video, such as the storyline corresponding to event A and the storyline corresponding to event B.

[0072] illustrative reference Figure 3The server generates storyline 3 for the original video 1 based on the selected short videos. The terminal displays the storyline interface for storyline 3, which includes a playback area for story clips and a storyline overview area. The overview area contains multiple story clips arranged in sequence, each corresponding to a target secondary creation video. Users can click on a specific story clip in the playback area to play it.

[0073] In summary, the method provided in this embodiment, when there are multiple secondary creation videos (short videos) obtained from the original drama video clips, filters out the target secondary creation video that belongs to the key plot segment from the multiple secondary creation videos, and generates the plot storyline of the original drama video based on the target secondary creation video. It provides a technical solution for displaying the video plot based on secondary creation videos as the main visual element. Since the presentation of short videos is richer and more intuitive, it can provide a much more diverse way of displaying the plot than a simple text and image storyline.

[0074] Furthermore, since different derivative videos may originate from different authors' editing of exciting plot segments, they can more accurately express the plot storyline that appeals to the general public, avoiding the plot storyline being limited by the thinking of individual staff members or enthusiastic netizens.

[0075] Figure 4 A flowchart illustrating a video story visualization method provided in an exemplary embodiment of this application is shown. This embodiment illustrates the application of this method in a server 140. The method includes:

[0076] Step 302: Obtain the original episode video and multiple secondary creation videos. The secondary creation videos are videos edited from the original episode video.

[0077] Original episode videos can be in the form of movies, TV series, documentaries, animations, etc. Original episode videos are relatively long, for example, exceeding a first threshold. This first threshold is 10 minutes, 30 minutes, etc. Original episode videos can be referred to as "long videos." The production company of the original episode video can be a film or television production company.

[0078] Derivative videos are videos created by editing original episode clips. For example, one or more video segments from an original episode can be edited into a single short video. The creators of derivative videos can be independent media creators, pre-production promotional organizations for the series, or enthusiastic netizens. The creators of derivative videos may be different from those of the original episode clips.

[0079] In some embodiments, the original episode videos and the derivative videos are on the same network platform, and the server of the network platform obtains the original episode videos and multiple derivative videos from the database of the network platform. In other embodiments, the original episode videos and all or part of the derivative videos are on different network platforms, and the server obtains the original episode videos and multiple derivative videos through web crawling tools.

[0080] Optionally, the server also filters multiple derivative videos. This filtering process includes, but is not limited to: filtering derivative videos whose content is inferior to the first condition, filtering derivative videos whose video clarity is inferior to the second condition, filtering derivative videos of multi-episode mashup types, etc.

[0081] Step 304: Obtain the first video data of the original drama series video and the second video data of the secondary creation video;

[0082] The original episode video has first video data, which includes data related to the consumption process of the original episode video. Illustratively, the consumption process-related data includes at least one of the following: the number of bullet comments, the number of likes, the number of favorites, and the number of skipped views corresponding to a specific playback time.

[0083] Secondary creation videos possess secondary video data, which includes data related to the production and / or consumption process of the secondary creation videos. Illustratively, data related to the production process includes at least one of the following: author level, number of followers, number of videos produced, and number of online platforms participated in. Data related to the consumption process includes at least one of the following corresponding to the playback time: number of bullet comments, number of likes, number of favorites, and number of views redirected to the next playback point.

[0084] Step 306: Based on the first video data and the second video data, calculate the plot key index at multiple playback time points in the original drama video;

[0085] The server can calculate the plot key index for multiple playback time points in the original episode video based on at least one of the first video data and the second video data.

[0086] This embodiment illustrates the example of a server simultaneously performing calculations based on both first and second video data. For example... Figure 5 As shown, this step may include the following three steps:

[0087] 306a, Based on the start and end times of the secondary creation video, determine multiple playback time points on the playback progress bar of the original drama video;

[0088] Since the derivative videos are all based on clips from the original episodes, the server uses video content understanding technology, such as video frame pattern recognition, to match each derivative video with the original episode video, thus obtaining the start and end times of each derivative video on the playback progress bar of the original episode video.

[0089] For example, taking the video series "Suspense Series A" as an example, the analysis data obtained includes the following table:

[0090] Table 1

[0091]

[0092]

[0093] The server uses these start and end times as multiple playback time points to be calculated on the playback progress bar of the original episode video.

[0094] Optionally, the server may also select multiple playback time points according to other strategies, such as selecting multiple playback time points according to a fixed step size, or selecting multiple playback time points according to a random strategy. This embodiment does not limit this.

[0095] 306b, for each playback time point, obtain the first video data and the second video data corresponding to that playback time point;

[0096] For each playback time point, the server filters out the first video data corresponding to that playback time point and the second video data corresponding to that playback time point.

[0097] The first video data includes the number of bullet comments in the original video series, and the second video data includes, for example, the number of derivative videos.

[0098] For the first video data, the original episode video has many bullet comments, each with its own timestamp. Based on the playback timestamp of each bullet comment, the bullet comments corresponding to playback time point A are filtered out, and the number of bullet comments corresponding to playback time point A is calculated.

[0099] For the second video data, the server filters out the secondary creation videos that include playback time point A in the playback interval, sums up the secondary creation videos corresponding to playback time point A, and calculates the number of secondary creation videos corresponding to playback time point A.

[0100] It should be noted that the first video data can also cover other types, such as the number of comments and likes; the second video data can also cover other types, such as the number of followers of the author of the derivative video, etc., which will not be elaborated on in this embodiment.

[0101] 306c calculates the plot key index at the playback time by weighted summation of the first and second video data corresponding to the playback time point.

[0102] For each playback time point, the first video data and the second video data corresponding to the playback time point are weighted and summed to calculate the plot key index of the playback time point.

[0103] For illustrative purposes, the plot key index at the playback time is calculated as: number of bullet comments * a + number of short videos * b. Here, a and b are preset weights. For illustrative purposes, a + b = 1.

[0104] Step 308: Based on the plot key index at multiple playback time points, select target secondary creation videos containing key plot segments from the original episode videos;

[0105] Based on plot key indices that meet local peak conditions at multiple playback time points, target derivative videos containing key plot segments from the original episodes are selected. The local peak condition, in the time dimension, detects whether the plot key index at a playback time point belongs to the peak plot key index within a local time period. For example, the local peak condition includes: the current peak plot key index is greater than other plot key indices within a predetermined preceding time period, and the current peak plot key index is greater than other plot key indices within a predetermined following time period, and the magnitude of the difference exceeds a threshold.

[0106] This step may include, for example, the following sub-steps: Figure 6 As shown:

[0107] 308a, using the time axis corresponding to the playback time point as the first coordinate axis and the index axis corresponding to the key plot index as the second coordinate axis, constructs comparative information of the key plot index. The comparative information includes any one of function curves, histograms and discrete point sequences.

[0108] like Figure 5 As shown, a discrete point sequence of plot key indices is constructed using the time axis corresponding to the playback time point as the x-axis and the index axis corresponding to the plot key index as the y-axis. Taking the original episode video with a length of 42:54 minutes as an example, there are multiple peak plot key indices in this discrete point sequence that satisfy the local peak condition.

[0109] 308b, based on the comparison information of key plot indices, select the peak key plot indices that meet the local peak conditions;

[0110] 308c, determining key plot segments based on the playback time points corresponding to the peak plot key index;

[0111] To illustrate, for a playback time point corresponding to a peak plot key index, determine the start and end time points closest to that playback time point, and determine the plot segment between the start and end time points as the plot key segment.

[0112] Alternatively, for two adjacent playback time points located at a local peak, determine the nearest start and end time points; and identify the plot segment between the start and end time points as key plot segments.

[0113] Alternatively, the playback time point corresponding to the peak plot key index can be determined as the start time point, and the playback time point of the next (or the i-th) plot key index adjacent to the peak plot key index can be determined as the end time point. The plot segment between the start time point and the end time point can be determined as the plot key segment.

[0114] Alternatively, the playback time point corresponding to the peak plot key index can be determined as the end time point, and the playback time point of the preceding (or i-th) plot key index adjacent to the peak plot key index can be determined as the start time point. The plot segment between the start time point and the end time point can be determined as the plot key segment. Here, i is an integer greater than 1.

[0115] For example, taking the video series "Suspense Series A" as an example, the key plot segments obtained are shown in Table 2 below:

[0116] Table 2

[0117]

[0118]

[0119] 308d, filtering out target derivative videos that contain key plot segments within the playback range.

[0120] This is illustrative; there are multiple key plot points, each corresponding to its own target-driven derivative video. Multiple derivative videos can correspond to the same key plot point.

[0121] Step 310: Based on the target secondary creation video, extract the names of the key plot segments for each plot segment;

[0122] A plot segment name is a textual summary that describes a key plot point. Optionally, each plot segment has its own plot segment name.

[0123] For each key plot segment, based on one or more target-related secondary creation videos corresponding to that key plot segment, the plot segment name of that key plot segment is extracted.

[0124] To illustrate, when at least two derivative videos correspond to the same key plot segment, the video titles of these two videos are clustered to obtain at least two candidate plot segment names. Based on plot keywords in the video titles, these candidate plot segment names are scored. Based on the scoring results, the plot segment name corresponding to the key plot segment is determined from these two candidate names.

[0125] For example, if a key plot segment corresponds to 120 short videos, the titles of these 120 short videos are clustered to obtain 3 candidate plot segment names. Then, the frequency of each word segment in the video titles of these 3 candidate plot segment names is calculated. Based on this frequency and the importance weight of each word segment, a weighted score is calculated for the 3 candidate plot segment names. The candidate plot segment name with the highest score is selected as the plot segment name corresponding to the current key plot segment.

[0126] As an illustration, when a key plot segment corresponds to one target derivative video, the title of that target derivative video is used as the plot segment name of the key plot segment; or, when the same key plot segment corresponds to at least two target derivative videos, one of the video titles of the at least two target derivative videos is randomly selected as the plot segment name of the key plot segment.

[0127] For example, if a key plot segment corresponds to only one short video, then the title of that short video is used as the plot segment name corresponding to the current key plot segment; or if a key plot segment corresponds to only two short videos, then one of the titles of those two short videos is randomly selected as the plot segment name corresponding to the current key plot segment.

[0128] This embodiment does not limit the method of generating plot segment names. In some embodiments, plot segment names can also be generated based on a pre-trained text generation model. For example, the video titles of multiple short videos corresponding to a key plot segment can be input into the text generation model, and the text generation model can output the plot segment name corresponding to the key plot segment.

[0129] Step 312: Group the plot segment names of key plot segments according to the plot line, and generate a plot story line of the original drama video by taking the target secondary creation video corresponding to the key plot segments in the same group.

[0130] This is illustrative; the plot can be divided according to characters, plot type, or both. This embodiment does not limit the method of plot division. In different dramas, the plot may also be divided according to timeline, protagonists / villains, a particular sect, or a specific location. This embodiment uses the example of a plot including characters and / or plot type for illustration.

[0131] Plot characters are those who appear in key parts of the plot. Different plot characters have relationships with each other, such as friends, father and son, husband and wife, college classmates, etc.

[0132] The plot type is a classification of plot creation information. For example, plot types include romance, suspense, music, musical, protagonist, antagonist, martial arts, treasure hunting, tomb raiding, crime investigation, etc. This embodiment does not limit the specific form of the plot type.

[0133] This step includes at least one of the following steps:

[0134] • Group the plot segment names of key plot points according to the plot characters, and generate a plot storyline for the original drama video by taking the target secondary creation video corresponding to the key plot segments belonging to the same plot character.

[0135] For example, multiple key plot segments belonging to the same character can be used to create a storyline for the original drama video, either by arranging the playback times from front to back, or by arranging the timeline within the drama from front to back.

[0136] • Group the plot segment names of key plot points according to plot type, and generate a plot storyline for the original episode video by plot segments belonging to the same plot type.

[0137] For example, multiple target secondary creation videos corresponding to key plot segments belonging to the same plot type can be generated into a plot storyline of the original drama video in chronological order of playback time, or in chronological order of the timeline within the drama video.

[0138] • Group the plot segment names of key plot points according to "plot characters and plot type", and generate a plot storyline for the original episode video by plot key segments belonging to the same "plot characters and plot type".

[0139] For example, multiple target secondary creation videos corresponding to key plot segments belonging to the same "plot character and plot type" can be generated into a plot storyline of the original drama video in chronological order of playback time, or in chronological order of the timeline in the drama.

[0140] Regarding the plot and characters:

[0141] The server also needs to identify candidate characters from the original episode videos.

[0142] In some embodiments, the server performs named entity recognition on the video title or key plot segment names of the target derivative video to obtain character entity recognition results; based on the character entity recognition results, it identifies, mines, or recognizes the plot characters. For example, if 10 character names are identified, the 3 character names that appear most frequently are selected as the main plot characters.

[0143] In some embodiments, the server stores a pre-built entity dictionary of characters in the TV series. Based on this pre-built entity dictionary, the server performs similar entity recognition on the names of key plot segments to obtain similar entity recognition results. Based on these results, the server identifies, mines, or recognizes characters in the plot. For example, if the pre-built entity dictionary contains the protagonist "Master Qingfeng," then "Old Man Qingfeng" and "Grandmaster Qingfeng," which are similar to "Master Qingfeng," are identified as the same character in the plot.

[0144] It should be noted that the aforementioned named entity recognition and similar entity mining techniques can be used in combination.

[0145] Regarding the plot structure, including the plot type:

[0146] In some embodiments, the aforementioned plot type is configured by the staff.

[0147] In some embodiments, an unsupervised method is used to collect textual statistics on the names of key plot segments. These statistics can include word frequency information, text ranking information, etc. Based on this textual statistics, the plot type is identified. For example, if the word "emotion" frequently appears in the video title, the plot type is determined to be a romance; similarly, if the word "ancient tomb" frequently appears in the video title, the plot type is determined to be a tomb raiding. The plot type keywords can be obtained from a pre-defined thesaurus.

[0148] In some embodiments, based on a supervised method, the plot type keywords in the plot segment names of key plot segments are combined with n candidate plot types to obtain n combined text pairs; the n combined text pairs are input into a matching model for scoring; and plot types are mined based on the scoring results of the n combined text pairs.

[0149] Where n is a positive integer, and the matching model is a machine learning model used to identify the degree of matching between the input text and the candidate plot type.

[0150] As an illustration, Table 3 shows the correspondence between the names of the extracted plot segments, the characters in the plot, and the plot types.

[0151] Table 3

[0152]

[0153]

[0154] As an illustration, for the same storyline, the server can also use a summary text generation model to generate summaries of the names of multiple key plot segments belonging to the storyline, thus obtaining the storyline name; or, the server can cluster the keywords in the names of multiple key plot segments belonging to the storyline, and combine the frequently occurring keywords into the storyline name; or, the storyline name can be manually generated by staff.

[0155] As an illustration, when there are at least two target derivative videos corresponding to the same key plot segment, a target derivative video is selected from the at least two target derivative videos based on the scoring dimension, and is used as the target derivative video corresponding to the key plot segment in the plot storyline, that is, the derivative video that is finally played for that key plot segment.

[0156] In summary, the method provided in this embodiment, when there are multiple secondary creation videos (short videos) obtained from the original drama video clips, filters out the target secondary creation video that belongs to the key plot segment from the multiple secondary creation videos, and generates the plot storyline of the original drama video based on the target secondary creation video. It provides a technical solution for displaying the video plot based on secondary creation videos as the main visual element. Since the presentation of short videos is richer and more intuitive, it can provide a much more diverse way of displaying the plot than a simple text and image storyline.

[0157] Furthermore, since different derivative videos may originate from different authors' editing of exciting plot segments, they can more accurately express the plot storyline that appeals to the general public, avoiding the plot storyline being limited by the thinking of individual staff members or enthusiastic netizens.

[0158] Figure 8 A flowchart illustrating a video story visualization method provided in an exemplary embodiment of this application is shown. The method can be executed by a terminal and includes:

[0159] Step 602: Display the storyline page of the first storyline of the original episode video. The storyline page includes introductory information for at least two key plot segments belonging to the first storyline.

[0160] A storyline page is a user interface used to display the plot storyline of the original episode video. For example, one plot storyline is displayed using one storyline page. Alternatively, multiple plot storylines can be displayed using the same storyline page. This embodiment illustrates this by showing one plot storyline using one storyline page.

[0161] A single original episode video can have one or more storylines. In the case of multiple storylines, the first storyline is one of them. For example, the first storyline might be the most important one; it might be the one that best matches the user's search keywords; or it might be any one of the multiple storylines or a random one.

[0162] illustrative reference Figure 9 The storyline page 10 includes a scene playback area 11 and a story introduction area 12. The story introduction area 12 displays the first storyline, "Storyline 1: Li XX Investigates the Cause of Death of His Master Lin XX," and introduction information 13 for several key scenes belonging to this first storyline. Taking the second key scene as an example, the introduction information 13 for this key scene includes the following:

[0163] • The title of a key plot segment is "Reunion with Old Classmate He XX";

[0164] • The key plot segment is marked "Segment 02";

[0165] • The duration of the derivative videos corresponding to key plot segments is “05:36”;

[0166] The character "He xx" appears in key plot segments;

[0167] • The key plot points are tagged with "High Energy".

[0168] It should be noted that, in addition to the introductory information of at least two key plot segments, the first plot storyline may also include other plot introduction information, such as text, pictures, audio, etc. This embodiment does not limit the specific form of the plot introduction information also included in the first plot storyline.

[0169] Step 604: Receive a trigger operation that provides introductory information for the first of at least two key plot points;

[0170] The first key plot point is either one of at least two key plot points, or the first key plot point is the one that the user is interested in among at least two key plot points.

[0171] The trigger operation can be at least one of the following: single click, double click, long press, swipe, pressure touch, hover touch, eye gaze, gesture, or motion control.

[0172] As an illustration, if a user is interested in the first of at least two key plot points, they can click on the first key plot point. Correspondingly, the terminal receives a trigger action from the user regarding the description information of the first key plot point out of the at least two key plot points.

[0173] Step 606: In response to the trigger operation, play the secondary creation video corresponding to the first key plot segment. The secondary creation video is a video edited from the original episode video.

[0174] In some embodiments, in response to a triggering operation, the user is redirected from the storyline page to another video playback page, where a secondary creation video corresponding to the first key plot segment is played.

[0175] The secondary creation video corresponding to the first key plot segment is the target secondary creation video corresponding to the first key plot segment mentioned in the above embodiments. When there are multiple target secondary creation videos corresponding to the first key plot segment, the secondary creation video played here is usually one that has been filtered out from the multiple target secondary creation videos for the first key plot segment.

[0176] In some embodiments, the storyline page includes a plot segment playback area, and in response to a trigger operation, a secondary creation video corresponding to a first key plot segment is played in the plot segment playback area of ​​the storyline page.

[0177] illustrative reference Figure 9 After the user clicks on the first key plot segment "Segment 02", the secondary creation video corresponding to the first key plot segment "Segment 02" will be played in the plot segment playback area 11 on the storyline page.

[0178] In summary, the method provided in this embodiment, when multiple secondary creation videos (short videos) obtained from editing the original drama series video exist, filters out the target secondary creation video that belongs to the key plot segment from the multiple secondary creation videos, and generates the plot storyline of the original drama series video based on the target secondary creation video. This provides a technical solution for displaying the video plot based on secondary creation videos as the main visual element. Since the presentation of short videos is richer and more intuitive, it can provide a much more diverse way of displaying the plot than a simple text and image storyline.

[0179] Figure 10 A flowchart illustrating a video story visualization method provided in an exemplary embodiment of this application is shown. The method can be executed by a terminal and includes:

[0180] Step 602a: Display the entry point for viewing the first storyline of the original episode video;

[0181] The terminal runs a client program, which can be an online video program, a browser program, a short video program, a microblogging program, etc.

[0182] The client application displays the first storyline of the original episode video as a viewing entry point. This entry point can be a graphical user interface (UI) element such as a button, menu item, or search bar.

[0183] Schematic, this step can adopt at least one of the following four possible designs:

[0184] Possible design one

[0185] Assuming the first storyline corresponds to the first main character; this step includes:

[0186] The character introduction interface displays the profile of the primary story character, including access points to view the corresponding storyline. Optionally, if the primary story character has multiple storylines, the character introduction interface can include all of them.

[0187] refer to Figure 11 The terminal displays the character introduction interface 20 for the first storyline character, "Li xx". This character introduction interface 20 includes two storylines related to "Li xx":

[0188] Storyline 1: Li XX investigates the cause of death of his mentor, Lin XX.

[0189] Storyline 2: Fourteen years later, the investigation into the Mai XX case is reopened.

[0190] If there are multiple storylines related to the main character "Li xx", multiple cards can be used to display these storylines, with each storyline corresponding to one card. Users can view different storyline cards by swiping up and down. When a user clicks on a storyline card (the entry point to view the storyline), they will be taken to the corresponding storyline page.

[0191] As an illustration, the entry point for viewing the storyline displays at least one of the following information:

[0192] • The name of the storyline;

[0193] • The number of key plot points belonging to the current storyline;

[0194] • Brief introductions to selected, high-energy, or exciting key plot points from the current storyline.

[0195] As an example, when displaying the character introduction screen for the first story character, the following methods can also be used:

[0196] Displays a character relationship diagram of the original episode video, which includes the main character and the relationships between other characters in the show.

[0197] In response to the selection of the first story character, the character introduction interface of the first story character is displayed.

[0198] refer to Figure 11 The terminal displays a character relationship diagram 24 containing the original episode video. Taking character relationship diagram 24 as an example, which includes Li xx, He xx, and Ma xx: Li xx and He xx are university classmates, and Li xx and Ma xx are friends. If the user selects Li xx, the terminal displays Li xx's character introduction interface; if the user selects He xx, the terminal displays He xx's character introduction interface; if the user selects Ma xx, the terminal displays Ma xx's character introduction interface.

[0199] Possible Design Two

[0200] Assuming the first storyline corresponds to the first story type, this step includes:

[0201] The original episode video's storyline aggregation interface displays a storyline aggregation interface that includes at least two different storylines of different plot types. The storylines of at least two different storylines of different plot types include the viewing entry point for the storyline corresponding to the first plot type.

[0202] refer to Figure 12 Assuming the original episode video includes at least four storylines:

[0203] Storyline 1: "Li XX investigates the cause of death of his mentor, Lin XX";

[0204] Storyline 2: "The investigation into the Mai XX case is reopened fourteen years later";

[0205] Storyline 3: "Ma XX suddenly dies in prison";

[0206] Storyline 4: "Who is the mole?"

[0207] These four storylines can then be aggregated and displayed in a single storyline aggregation interface, which serves as the entry point for viewing the storylines. If there are multiple storylines, some can be collapsed. By clicking the "View More Storylines" button, the collapsed remaining storylines can be viewed.

[0208] Possible design three

[0209] Assuming the first storyline is the one that matches the search keywords; this step includes:

[0210] Upon receiving a search keyword, a search results interface matching the search keyword is displayed, which includes access points to view storylines matching the search keyword.

[0211] refer to Figure 13 The user enters the search keyword "suspense drama series A" in search box 31. The terminal displays a search results interface matching the search keyword "suspense drama series A," which includes access points to view the plot storylines matching the search keyword. For example, the search results interface might include plot storyline 1 for the character "Li xx." If multiple matching plot storylines exist, they can be displayed separately as multiple storyline cards. Figure 12 If the user slides the storyline card 32 of storyline 1 to the left, they can view the storyline card of storyline 2.

[0212] Possible Design Four

[0213] The first storyline is the storyline to which the target derivative video belongs; this step includes:

[0214] The video playback interface displays the target's derivative video, which includes an entry point to view the storyline to which the target's derivative video belongs.

[0215] refer to Figure 14The terminal displays a short video playback interface. Below the "Today's Recommendations" section, there is a playback entry for the short video "High-Energy Clip from 'XX Storm'". There is also a "View More Videos Recommended for This Storyline" button for the storyline associated with the short video "High-Energy Clip from 'XX Storm'". After clicking the "View More Videos Recommended for This Storyline" button, the user can view other short videos within the same storyline as "High-Energy Clip from 'XX Storm'".

[0216] It should be noted that the different designs mentioned above can also be combined and implemented. For example... Figure 15 As shown, when a user enters the search keyword "suspense drama series A" into the browser's search entry, the terminal displays the search results page 30 for the original video drama series "suspense drama series A". The search results page 30 includes a brief introduction to "suspense drama series A", access to each series, and storyline aggregation cards grouped according to different plot characters.

[0217] After the user selects the storyline aggregation card for the plot character "Li xx", the terminal displays the character introduction page 20 for "Li xx". This character introduction page 20 includes a character relationship diagram for the plot character "Li xx", storyline 1, and storyline 2. When the user clicks the view entry for storyline 1, the storyline page 10 corresponding to storyline 1 for the plot character "Li xx" is displayed. This storyline page 10 includes introductory information for several key plot segments.

[0218] Since the character "Li xx" has two storylines, the current display is the storyline page for storyline 1. In response to a swipe operation (such as swiping left), the terminal will switch the display of the storyline page for the first storyline "storyline 1" to the storyline page for the second storyline "storyline 2".

[0219] Step 602b: In response to the trigger action on the view entry, the storyline page includes introductory information for at least two key plot segments belonging to the first storyline;

[0220] Information about at least two key plot points belonging to the first storyline can be displayed in chronological order. If the current page cannot display information about all key plot points, you can swipe up or down to view information about other key plot points.

[0221] Step 604: Receive a trigger operation that provides introductory information for the first of at least two key plot points;

[0222] In response to the triggered action, a secondary creation video corresponding to the first key plot segment is played in the plot segment playback area.

[0223] The first key plot point is either one of at least two key plot points, or the first key plot point is the one that the user is interested in among at least two key plot points.

[0224] The trigger operation can be at least one of the following: single click, double click, long press, swipe, pressure touch, hover touch, eye gaze, gesture, or motion control.

[0225] As an illustration, if a user is interested in the first of at least two key plot points, they can click on the first key plot point. Correspondingly, the terminal receives a trigger action from the user regarding the description information of the first key plot point out of the at least two key plot points.

[0226] Step 606: In response to the trigger operation, play the secondary creation video corresponding to the first key plot segment. The secondary creation video is a video edited from the original episode video.

[0227] In some embodiments, in response to a triggering operation, the user is redirected from the storyline page to another video playback page, where a secondary creation video corresponding to the first key plot segment is played.

[0228] In some embodiments, the storyline page includes a plot segment playback area, and in response to a trigger operation, a secondary creation video corresponding to a first key plot segment is played in the plot segment playback area of ​​the storyline page.

[0229] illustrative reference Figure 9 Users can select different key plot segments in the plot introduction area below, and then view the selected key plot segments in the plot segment playback area 11 above.

[0230] In summary, the method provided in this embodiment, when multiple secondary creation videos (short videos) obtained from editing the original drama series video exist, filters out the target secondary creation video that belongs to the key plot segment from the multiple secondary creation videos, and generates the plot storyline of the original drama series video based on the target secondary creation video. This provides a technical solution for displaying the video plot based on secondary creation videos as the main visual element. Since the presentation of short videos is richer and more intuitive, it can provide a much more diverse way of displaying the plot than a simple text and image storyline.

[0231] In a specific example, taking the original video series as long-form videos and the derivative videos as short-form videos, refer to... Figure 16 The processing flow of the aforementioned server includes two stages: the plot exploration stage 92 and the plot development stage 94.

[0232] Plot exploration stage 92

[0233] The plot analysis relies on long-form video resources from the original TV series and short-form video resources obtained through secondary creation (editing, splicing) based on these long-form video resources. Long-form video resources include video data from the original TV series and corresponding consumption data (such as user comments, bullet comments, etc.); short-form video resources include secondary-created short-form video data and related production / consumption data (author level, author posts, number of author followers, number of likes on short-form videos, number of comments on short-form videos, etc.).

[0234] The specific mining steps include plot detection (extraction of key plot fragments) and plot extraction:

[0235] Plot detection

[0236] "Short-to-Long": Utilizing video content understanding capabilities (primarily video frame pattern recognition), this method recalls and partially matches multiple short video resources and preliminary matching information (the approximate start and end times of the short video matching the long video, i.e., the original episode's time interval). Based on the signals from the short video resources, low-quality resources such as multi-episode mashups are initially filtered out. The "short-to-long" data obtained using the IP drama series "Anti-gangster XX" as an example is an example.

[0237] For each episode of long video content, the "production signal" and "consumption signal" at each time point are statistically analyzed. Peak time segments are obtained through peak detection strategies, which are the key plot time segments of each episode. The "production signal" at each time point is statistically analyzed by accumulating the original episode time intervals corresponding to multiple candidate short production videos, thus obtaining the distribution of the number of short production videos on the entire timeline. Similarly, for "consumption signals" (such as user comments), a histogram distribution statistical analysis can also be performed on the timeline.

[0238] Two signals are weighted and accumulated. A strategy (such as determining if a signal value is greater than the signal values ​​before and after a certain time interval, and also greater than a certain threshold) is used to filter multiple time intervals corresponding to local peaks in the signal, resulting in multiple key plot points. By accumulating the short video resources associated with each time point within these time segments, the short video resources related to each time segment can be obtained.

[0239] Story Extraction

[0240] From the time segments of multiple key plot points and corresponding short video resources obtained from plot detection, structured information such as plot name, characters, and plot type is mined and extracted.

[0241] Character mining: Based on named entity recognition technology and a pre-built dictionary of characters in TV series (name, nickname), mining is performed from information such as titles and descriptions of relevant short video resources.

[0242] Story title mining: Short video titles related to key story segments can be clustered. For the same key story segment, candidate short video titles are scored based on story characters and keywords, and the short video title with the highest score is selected as the story title for that key story segment; alternatively, a text generation model can be trained to generate story titles.

[0243] Plot type mining: Unsupervised methods use statistical word frequency or Text Rank methods to mine type keywords (such as romance, shooting) from the titles and descriptions of relevant short videos as plot type information; supervised methods use manual specification of multiple plot line types, such as character story lines (such as "Li Chengyang investigates the cause of death of his master Lin Han"), and use a matching model to score the text pairs composed of plot names and candidate plot types to select the best plot type.

[0244] Plot development stage 94

[0245] The plot detection step generates one or more short video resources corresponding to each key plot segment. From these candidate short video resources, the best matching short video can be further selected as the descriptive video for the plot. The selection method is based on a comprehensive scoring and ranking of the short video title and plot name matching degree, the short video mapping time and plot detection time matching degree, and the short video's own quality (whether the content is clear, the author's level, etc.), and the short video with the highest score is selected.

[0246] By following the three steps above, key plot time segments of each episode can be extracted, along with corresponding structured plot content information (plot name, characters, plot type, etc.) and the best matching short videos, which can then be used to construct the plot outline and satisfy search results.

[0247] Figure 17 A block diagram of a video story visualization apparatus provided in an exemplary embodiment of this application is shown. The video story visualization apparatus includes:

[0248] The video acquisition module 1720 is used to acquire original TV series videos and multiple secondary creation videos, wherein the secondary creation videos are videos edited from the original TV series videos.

[0249] The plot mining module 1740 is used to filter out target secondary creation videos containing key plot segments of the original TV series videos from the multiple secondary creation videos.

[0250] The plot development module 1760 is used to generate a plot storyline of the original drama video based on the target secondary creation video, and the plot storyline includes at least two of the target secondary creation videos.

[0251] In some embodiments, the plot mining module 1740 is used to calculate the plot key index of multiple playback time points in the original drama video based on at least one of the first video data and the second video data; and to filter out secondary creation videos containing plot key segments of the original drama video based on the plot key index of the multiple playback time points.

[0252] The first video data includes data related to the consumption process of the original drama video, and the second video data includes data related to the production and / or consumption process of the secondary creation video.

[0253] In some embodiments, the plot mining module 1740 is used to obtain the first video data and the second video data corresponding to each playback time point; and to calculate the plot key index of the playback time point by weighted summation of the first video data and the second video data corresponding to the playback time point.

[0254] In some embodiments, the plot mining module 1740 is used to construct comparative information of the plot key index using the time axis corresponding to the playback time point as the first coordinate axis and the index axis corresponding to the plot key index as the second coordinate axis. The comparative information includes any one of function curves, histograms, and discrete point sequences. Based on the comparative information of the plot key index, peak plot key indices that meet local peak conditions are selected. The key plot segments are determined based on the playback time point corresponding to the peak plot key index. Target secondary creation videos containing the key plot segments in the playback interval are selected.

[0255] In some embodiments, the plot development module 1760 is used to mine the plot segment name of each key plot segment based on the target secondary creation video;

[0256] The plot segment names of the key plot segments are grouped according to the plot line. The target secondary creation videos corresponding to the key plot segments belonging to the same group are generated as a plot story line of the original drama video.

[0257] In some embodiments, the plot development module 1760 is configured to, when there are at least two target secondary creation videos corresponding to the same key plot segment, cluster the video titles of the at least two target secondary creation videos to obtain at least two candidate plot segment names; score the at least two candidate plot segment names based on the plot keywords in the video titles; and determine the plot segment name corresponding to the key plot segment from the at least two candidate plot segment names based on the scoring results.

[0258] In some embodiments, the plotline includes plot characters;

[0259] The plot development module 1760 is used to group the plot segment names of the key plot segments according to the plot characters, and generate a plot storyline of the original drama video by taking the target secondary creation video corresponding to the key plot segments belonging to the same plot character.

[0260] The plot development module 1760 is used to perform named entity recognition on the plot segment names of the key plot segments to obtain character entity recognition results; identify the plot characters based on the character entity recognition results; and / or; perform similar entity recognition on the plot segment names of the key plot segments based on a pre-built drama character entity dictionary to obtain similar entity recognition results; and identify the plot characters based on the similar entity recognition results.

[0261] In some embodiments, the plotline includes a plot type;

[0262] The plot development module 1760 is used to group the plot segment names of the key plot segments according to the plot type, and generate a plot storyline of the original episode video by plot key segments belonging to the same plot type.

[0263] In some embodiments, the plot development module 1760 is used to collect text statistics of the plot segment names of the key plot segments; to mine the plot type based on the text statistics; and / or to combine the plot segment names of the key plot segments with n candidate plot types to obtain n combined text pairs; to input the n combined text pairs into a matching model for scoring; and to mine the plot type based on the scoring results of the n combined text pairs.

[0264] Where n is a positive integer, the matching model is a machine learning model used to identify the degree of matching between input text and candidate plot types.

[0265] In some embodiments, the plot development module 1760 is used for:

[0266] When there are at least two target derivative videos corresponding to the same key plot segment, a target derivative video is selected from the at least two target derivative videos based on the scoring dimension, and is used as the target derivative video corresponding to the key plot segment in the plot storyline.

[0267] Figure 18 A block diagram of a video story visualization apparatus provided in an exemplary embodiment of this application is shown. The video story visualization apparatus includes:

[0268] Display module 1820 is used to display the storyline page of the first plot storyline of the original TV series video, the storyline page including introductory information of at least two key plot segments belonging to the first plot storyline;

[0269] The human-computer interaction module 1840 is used to receive trigger operations for the introduction information of the first key plot segment in the at least two key plot segments, as well as other trigger operations, selection operations or human-computer interaction operations.

[0270] The display module 1820 is used to respond to the trigger operation and play a secondary creation video corresponding to the first key plot segment, wherein the secondary creation video is a video edited from the original episode video.

[0271] In some embodiments, the display module 1820 is used to display the viewing entry point for the first plot storyline of the original episode video;

[0272] In response to the triggered operation on the viewing entry, the storyline page of the first plot storyline of the original episode video is displayed.

[0273] In some embodiments, the first storyline is a storyline corresponding to the first character.

[0274] The display module 1820 is used to display the character introduction interface of the first plot character, and the character introduction interface includes the viewing entry of the plot story line corresponding to the first plot character.

[0275] In some embodiments, the display module 1820 is configured to display a character relationship diagram of the original episode video, the character relationship diagram including the relationship between the first plot character and other characters besides the first plot character; in response to the selection operation of the first plot character, the character introduction interface of the first plot character is displayed.

[0276] In some embodiments, the first storyline is a storyline corresponding to a first story type;

[0277] The display module 1820 is used to display the storyline aggregation interface of the original drama video. The storyline aggregation interface includes at least two storylines of different plot types, and the at least two storylines of different plot types include the viewing entry of the storyline corresponding to the first plot type.

[0278] In some embodiments, the first storyline is a storyline that matches the search keywords;

[0279] The display module 1820 is used to display a search results interface that matches the search keyword when the search keyword is received. The search results interface includes an entry point for viewing the storyline that matches the search keyword.

[0280] In some embodiments, the first storyline is the storyline to which the target derivative video belongs;

[0281] The display module 1820 is used to display the video playback interface of the target secondary creation video, and the video playback interface includes a viewing entry point for the plot storyline to which the target secondary creation video belongs.

[0282] In some embodiments, the storyline page includes a plot introduction area and a plot segment playback area, wherein the plot introduction area displays introduction information for at least two key plot segments belonging to the first plot storyline;

[0283] The display module 1820 is used to respond to the trigger operation and play a secondary creation video corresponding to the first key plot segment in the plot segment playback area.

[0284] In some embodiments, the introductory information of the first key plot segment includes at least one of the following:

[0285] The name of the plot segment in the first key plot point;

[0286] The identifier of the first key plot segment;

[0287] The duration of the secondary creation video corresponding to the first key plot segment;

[0288] The characters appearing in the first key plot segment;

[0289] The segment tag for the first key plot point.

[0290] Figure 19 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Typically, the computer device 1900 includes a processor 1920 and a memory 1940.

[0291] Processor 1920 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1920 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1920 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1920 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1920 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0292] The memory 1940 may include one or more computer-readable storage media, which may be non-transitory. The memory 1940 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1940 are used to store at least one instruction, which is executed by the processor 1920 to implement the method provided in the method embodiments of this application.

[0293] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the video plot visualization method provided in the above-described method embodiments.

[0294] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the video plot visualization method provided in the above method embodiments.

[0295] Optionally, this application also provides a computer program product containing instructions that, when run on a computer device, cause the computer device to perform the video plot visualization method described in the above aspects.

[0296] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0297] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0298] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for visualizing video storylines, characterized in that, The method includes: The original TV series video and multiple secondary creation videos are obtained, wherein the secondary creation videos are videos edited from the original TV series video; Based on the first video data and the second video data, the plot key index of multiple playback time points in the original drama video is calculated. The playback time point refers to the time point on the playback progress bar of the original drama video. The playback time point corresponding to each video frame in the secondary creation video is obtained by aligning the video frames of the secondary creation video with those of the original drama video. Based on the plot key index that meets the local peak condition in the multiple playback time points, target secondary creation videos containing plot key segments of the original drama video are selected. The local peak condition is the condition for detecting whether the plot key index of the playback time point belongs to the peak plot key index in a local time period in the time dimension. Based on the plot key index of the multiple playback time points, secondary creation videos containing plot key segments of the original TV series video are selected. The first video data includes data related to the consumption process of the original drama video, and the second video data includes data related to the production and / or consumption process of the secondary creation video; the data related to the production process includes at least one of the following: author level, number of fans, number of videos produced, and number of online platforms participated in; the data related to the consumption process includes at least one of the following: number of bullet comments, number of comments, number of likes, number of favorites, and number of jump playbacks corresponding to the playback time point.

2. The method according to claim 1, characterized in that, The calculation of plot key indices at multiple playback points in the original drama video based on the first video data and the second video data includes: For each playback time point, obtain the first video data and the second video data corresponding to the playback time point; The plot key index for the playback time point is calculated by weighted summation of the first video data and the second video data corresponding to the playback time point.

3. The method according to claim 1, characterized in that, The method of selecting target secondary creation videos containing key plot segments from the original TV series videos based on the plot key index that meets the local peak conditions at the multiple playback time points includes: Using the time axis corresponding to the playback time point as the first coordinate axis and the index axis corresponding to the key plot index as the second coordinate axis, a comparison information of the key plot index is constructed. The comparison information includes any one of function curves, histograms, and discrete point sequences. Based on the comparison information of the aforementioned key plot indices, peak key plot indices that meet the local peak conditions are selected. The key plot segments are determined based on the playback time points corresponding to the peak plot key index. Select target derivative videos whose playback range contains key plot segments.

4. The method according to any one of claims 1 to 3, characterized in that, The process of generating the original drama video's storyline based on the target secondary creation video includes: Based on the target, the names of the plot segments for each key plot segment are extracted from the secondary creation video; The plot segment names of the key plot segments are grouped according to the plot line. The target secondary creation videos corresponding to the key plot segments belonging to the same group are generated as a plot story line of the original drama video.

5. The method according to claim 4, characterized in that, The plot segment names for each key plot segment mined from the target secondary creation video include: When there are at least two target derivative videos corresponding to the same key plot segment, the video titles of the at least two target derivative videos are clustered to obtain at least two candidate plot segment names. Based on the plot keywords in the video title, the at least two candidate plot segment names are scored; Based on the scoring results, the plot segment name corresponding to the key plot segment is determined from the at least two candidate plot segment names.

6. The method according to claim 4, characterized in that, The plot outline includes plot characters and / or plot type; The process of grouping the plot segment names of the key plot points according to the plot outline, and generating a plot storyline for the original episode video by grouping the key plot segments belonging to the same group, includes: The plot segment names of the key plot segments are grouped according to the plot characters. The target secondary creation videos corresponding to the key plot segments belonging to the same plot character are generated as a plot storyline of the original drama video.

7. A visualization device for video storylines, characterized in that, The device includes: The video acquisition module is used to acquire original episode videos and multiple secondary creation videos, wherein the secondary creation videos are videos edited from the original episode videos. The plot mining module is used to calculate the plot key index at multiple playback time points in the original drama video based on the first video data and the second video data. The playback time point refers to the time point on the playback progress bar of the original drama video. The playback time points corresponding to each video frame in the secondary creation video are obtained by aligning the video frames of the secondary creation video with those of the original drama video. Based on the plot key index that meets the local peak condition among the multiple playback time points, target secondary creation videos containing plot key segments of the original drama video are selected. The local peak condition is the condition for detecting whether the plot key index at the playback time point belongs to the peak plot key index within a local time period in the time dimension. The plot development module is used to generate the plot storyline of the original drama video based on the target secondary creation video, and the plot storyline includes at least two of the target secondary creation videos; The first video data includes data related to the consumption process of the original drama video, and the second video data includes data related to the production and / or consumption process of the secondary creation video; the data related to the production process includes at least one of the following: author level, number of fans, number of videos produced, and number of online platforms participated in; the data related to the consumption process includes at least one of the following: number of bullet comments, number of comments, number of likes, number of favorites, and number of jump playbacks corresponding to the playback time point.

8. The apparatus according to claim 7, characterized in that, The plot mining module is used for: For each playback time point, obtain the first video data and the second video data corresponding to the playback time point; The plot key index for the playback time point is calculated by weighted summation of the first video data and the second video data corresponding to the playback time point.

9. The apparatus according to claim 7, characterized in that, The plot mining module is used for: Using the time axis corresponding to the playback time point as the first coordinate axis and the index axis corresponding to the key plot index as the second coordinate axis, a comparison information of the key plot index is constructed. The comparison information includes any one of function curves, histograms, and discrete point sequences. Based on the comparison information of the aforementioned key plot indices, peak key plot indices that meet the local peak conditions are selected. The key plot segments are determined based on the playback time points corresponding to the peak plot key index. Select target derivative videos whose playback range contains key plot segments.

10. The apparatus according to any one of claims 7 to 9, characterized in that, The plot development module is used for: Based on the target, the names of the plot segments for each key plot segment are extracted from the secondary creation video; The plot segment names of the key plot segments are grouped according to the plot line. The target secondary creation videos corresponding to the key plot segments belonging to the same group are generated as a plot story line of the original drama video.

11. The apparatus according to claim 10, characterized in that, The plot development module is used for: When there are at least two target derivative videos corresponding to the same key plot segment, the video titles of the at least two target derivative videos are clustered to obtain at least two candidate plot segment names. Based on the plot keywords in the video title, the at least two candidate plot segment names are scored; Based on the scoring results, the plot segment name corresponding to the key plot segment is determined from the at least two candidate plot segment names.

12. The apparatus according to claim 10, characterized in that, The plot outline includes plot characters and / or plot type; The plot development module is used for: The plot segment names of the key plot segments are grouped according to the plot characters. The target secondary creation videos corresponding to the key plot segments belonging to the same plot character are generated as a plot storyline of the original drama video. or, The plot segment names of the key plot segments are grouped according to the plot type, and the key plot segments belonging to the same plot type are used to generate a plot storyline for the original episode video; or, The plot segment names of the key plot segments are grouped according to the plot characters and the plot type. The key plot segments belonging to the same plot characters and plot type are used to generate a plot storyline for the original episode video.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is executed by the processor to enable the computer device to implement the video plot visualization method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to enable a device having the processor to implement the video plot visualization method as described in any one of claims 1 to 6.

15. A computer program product, characterized in that, The computer program product stores a computer program that is executed by a processor to enable a device having the processor to implement the video plot visualization method as described in any one of claims 1 to 6.

Images