Stroma viewing point generation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and particularly provides a plot viewing point generation method and device, equipment and a storage medium, and the method comprises the steps: determining bullet screen theme information which comprises bullet screen subject terms and time periods corresponding to the bullet screen subject terms; obtaining a target plot abstract, wherein a time period corresponding to the target plot abstract and a time period corresponding to the bullet screen subject term have an intersection; a target video clip is obtained, a time period corresponding to the target video clip and a time period corresponding to the bullet screen subject term have an intersection, a first picture set is extracted from the target video clip, and a first picture of the first picture set is obtained by performing frame extraction on the target video clip; and generating a plot watching point of the target video by using the bullet screen subject term, the target plot abstract and the target picture set. The plot phrases are mined by adopting the bullet screen topics, and the contents of the pictures in the video are added, so that the contents of the plot phrases are richer, and the quality of the plot phrases is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for generating plot highlights. Background Art

[0002] With the rapid development of Internet technology, there are a large number of film and television works in video applications. In order to facilitate users to quickly understand the plot of film and television works, or quickly locate the film and television works they are interested in, it is usually necessary to summarize and describe the most attractive plots or turning points in film and television resources (such as TV series and movies) to form plot highlights.

[0003] In the related art, a technician watches a film or TV work, or checks the script of the film or TV work, and sets one or more highlight plot points in the film or TV work based on the technician's understanding of the film or TV work.

[0004] The above process is cumbersome, resulting in low efficiency in generating plot highlights. In addition, due to subjective factors, the set plot highlights may differ greatly from the actual highlights of the film and television works, which in turn causes users to be unable to correctly understand the plot of the film and television works, affecting the dissemination effect of the film and television works. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a plot highlight generation method, device, equipment and storage medium to improve the efficiency of plot highlight generation and the quality of plot highlights, so that the plot highlights are more in line with the actual content of the film and television works, so as to help users understand the plot of the film and television works more correctly and improve the dissemination effect of the film and television works.

[0006] In a first aspect, the present application provides a method for generating plot highlights, the method comprising: determining barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words, the barrage theme words are phrases determined based on multiple barrage texts with an associated relationship, the time period corresponding to the barrage theme words is the time period in which multiple barrage texts appear in an initial video, and the initial video is the video watched when the user inputs the barrage text; obtaining a target plot summary, the time period corresponding to the target plot summary and the time period corresponding to the barrage theme words have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video; obtaining a target video clip, the time period corresponding to the target video clip and the time period corresponding to the barrage theme words have an intersection, and the time period corresponding to the target video clip is the time period in which the target video clip is in the initial video; extracting a first picture set from the target video clip, the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video clip; and generating plot highlights of the target video using the barrage theme words, the target plot summary, and the target picture set.

[0007] In a second aspect, the present application provides a plot highlight generation device, which includes: a barrage determination module, which is used to determine barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words, the barrage theme words are phrases determined based on multiple barrage texts with associated relationships, and the time periods corresponding to the barrage theme words are time periods in which multiple barrage texts appear in an initial video, and the initial video is the video watched when the user inputs the barrage text; a plot summary acquisition module, which is used to acquire a target plot summary, the time period corresponding to the target plot summary and the time period corresponding to the barrage theme words have an intersection, and the time period corresponding to the target plot summary The segment is the time period of the video clip corresponding to the target plot summary in the initial video; a video clip acquisition module is used to obtain the target video clip, the time period corresponding to the target video clip and the time period corresponding to the barrage theme word have an intersection, and the time period corresponding to the target video clip is the time period of the target video clip in the initial video; a first picture set extraction module is used to extract a first picture set from the target video clip, the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video clip; a plot highlight generation module is used to generate the plot highlights of the target video using the barrage theme words, the target plot summary and the target picture set.

[0008] In a third aspect, the present application provides an electronic device comprising: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by one or more processors, the one or more processors implement the plot highlight generation method as described in the first aspect above.

[0009] In a fourth aspect, the present application provides a storage medium, which may be a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for generating plot highlights as described in the first aspect above is implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method for generating plot highlights as described in the first aspect above is implemented.

[0011] Compared with the prior art, the technical solution provided by the embodiments of the present application has the following advantages:

[0012] The embodiment of the present application provides a method, device, equipment and storage medium for generating plot highlights, the method comprising: determining bullet screen theme information, wherein the bullet screen theme information includes bullet screen theme words and time periods corresponding to the bullet screen theme words, the bullet screen theme words are phrases determined based on multiple bullet screen texts with an associated relationship, the time period corresponding to the bullet screen theme words is the time period in which the multiple bullet screen texts appear in an initial video, and the initial video is the video watched when the user inputs the bullet screen text; obtaining a target plot summary, the time period corresponding to the target plot summary and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video; obtaining a target video clip, the time period corresponding to the target video clip and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target video clip is the time period in which the target video clip is located in the initial video; extracting a first picture set from the target video clip, the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video clip; and generating plot highlights of the target video using the bullet screen theme words, the target plot summary and the target picture set.

[0013] In the embodiment of the present application, starting from the bullet screen, the pictures in the plot summary and the video clips are analyzed to generate the final plot highlights. Since the bullet screen theme is obtained by analyzing the bullet screen text input by the user, the generated plot highlights are more in line with the interests of the public, and the quality of the plot highlights is improved. In the process of generating the plot highlights, the picture analysis of the video clips is introduced. The picture analysis is visual content, which makes the final plot highlights richer in content and further improves the quality of the plot highlights. In addition, this process avoids the step of manually analyzing the content of the bullet screen, improves the efficiency of generating plot highlights, and avoids the influence of human subjective factors on the plot highlights, further improving the quality of the plot highlights. Therefore, the quality of the plot highlights finally generated by the embodiment of the present application is higher, which can attract more users to watch the video content and improve the dissemination efficiency of film and television works. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0016] Figure 1 A flowchart of a plot highlight generation method provided in an embodiment of the present application;

[0017] Figure 2 A flowchart of another plot highlight generation method provided in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of the structure of a plot highlight generating device provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0022] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0023] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0025] The following is a detailed description of the plot phrase generation method provided by the present application in conjunction with the accompanying drawings and specific implementation methods.

[0026] Figure 1This is a flow chart of a plot highlight generation method in an embodiment of the present application. This embodiment is applicable to the case of generating exciting highlights. The method can be executed by a plot highlight generation device. The plot highlight generation device can be implemented in software and / or hardware. The plot highlight generation device can be configured in an electronic device. The electronic device can be any one of a smart phone, a laptop computer, a desktop computer, a portable computer, a server, etc.

[0027] like Figure 1 As shown, the plot highlight generation method provided in the embodiment of the present application mainly includes steps S101-S105.

[0028] S101. Determine the barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words. The barrage theme words are phrases determined based on multiple barrage texts with associated relationships. The time period corresponding to the barrage theme words is a time period in which multiple barrage texts appear in an initial video. The initial video is the video watched when the user inputs the barrage text.

[0029] Among them, barrage is an effect in which the real-time comments sent by users float across the screen in the form of subtitles during video playback. Barrage text refers to the comment content sent by users and displayed in the form of barrage. In other words, barrage text can refer to the real-time comment content sent by users, and the comment content can be composed of text and / or special symbols. Furthermore, when users send voice barrage while watching videos, the barrage text is obtained after audio recognition of the voice in the language barrage.

[0030] The barrage keywords are phrases determined based on multiple barrage texts with associated relationships. Among them, the barrage texts with associated relationships can be understood as the themes or topics discussed by multiple barrage texts are the same or similar. In other words, the barrage keywords are the common content or subject content of the barrage texts within a period of time. The barrage keywords are usually obtained by semantically analyzing multiple barrage texts using a pre-set algorithm. Specifically, a certain number of barrage texts or a period of time are collected, and the latent Dirichlet allocation algorithm (LDA) or the latent semantic analysis algorithm (LSA) can be used to perform semantic analysis on the collected barrage texts to obtain the barrage keywords.

[0031] The initial video is the video that the user is watching when inputting the barrage text. The initial video may be an episode of a TV series, a movie, a clip of a TV series or a movie, etc. During the playback of the initial video, multiple barrage texts sent by the user are received, and semantic analysis is performed on the multiple barrage texts to obtain one or more barrage theme words.

[0032] In an initial video, as the plot develops, different bullet screen keywords may exist at different plot stages. Therefore, there may be multiple bullet screen keywords in an initial video. For each bullet screen keyword, the plot highlight generation method provided in the embodiment of the present application can be used to generate its corresponding plot highlight.

[0033] Since the barrage theme words are obtained by aggregating multiple barrage texts, and since multiple barrage texts may be sent by one or more viewers at different time points, resulting in different sending time points for multiple barrage texts, when multiple viewers send barrage texts with the same topic or discussion point in the same initial video, multiple barrage texts form a barrage theme word. The time period in which multiple barrage texts of the barrage theme word appear in the initial video obtained by semantic analysis is the time period corresponding to the barrage theme word. The time period corresponding to the barrage theme word can be the time interval between the earliest sending time point and the latest sending time point of the multiple barrage texts that form the barrage theme word.

[0034] Exemplarily, the barrage keyword is obtained by semantic analysis of barrage text 1, barrage text 2, ..., barrage text N, and the subject or topic discussed by barrage text 1, barrage text 2, ..., barrage text N is the same. The i-th barrage text among the N barrage texts is sent the earliest, when the initial video is played to 10:35, and the j-th barrage text among the N barrage texts is sent the latest, when the initial video is played to 15:48. The other barrage texts among the N barrage texts are all sent within the time interval [10:35, 15:48]. Then the time period corresponding to the barrage keyword is [10:35, 15:48].

[0035] S102, obtaining a target plot summary, where a time period corresponding to the target plot summary overlaps with a time period corresponding to the bullet screen keyword, and the time period corresponding to the target plot summary is a time period of a video clip corresponding to the target plot summary in the initial video.

[0036] A plot summary refers to a text summary that summarizes the plot of a film or TV work. The plot summary in the embodiment of the present application may include a minute-level plot summary. Among them, the minute-level plot summary is a detailed description of the content of the episode in chronological order, usually accurate to what key events or dialogues occurred every minute. The minute-level plot summary is mainly used to help people quickly understand or review the main plot of a movie, TV series or video.

[0037] In practical applications, a plot summary usually refers to a minute-level plot summary of a TV series or a movie. The above target plot summary can be understood as a plot summary related to the bullet screen theme words. In other words, the plot described in the target plot summary is the same or similar to the theme content represented by the bullet screen theme words. Or the theme content represented by the bullet screen theme words is part of the plot content in the target plot summary.

[0038] Since each important plot or dialogue in the minute-level plot summary is recorded and the exact time of appearance of each plot or dialogue in the original video is marked (for example, "5th minute", "23rd minute 45 seconds"), it can help quickly locate a specific scene. The time period corresponding to the target plot summary is the time period when the important plot or dialogue included in the target plot summary appears in the original video.

[0039] In one possible implementation, a minute-level plot summary of the initial video is obtained, and a minute-level plot summary whose time period overlaps with the time period corresponding to the bullet-screen theme word is used as the target plot summary. In other words, the time period corresponding to the bullet-screen theme word overlaps with the time period corresponding to the target plot summary.

[0040] In one possible implementation, obtaining a target plot summary includes: taking the minimum value between a first end time point and a second end time point as the minimum time point, wherein the first end time point refers to the end time point of the time period corresponding to the barrage theme word, and the second end time point refers to the end time point of the time period corresponding to the plot summary; taking the maximum value between a first start time point and a second start time point as the maximum time point, wherein the first start time point refers to the start time point of the time period corresponding to the barrage theme word, and the second start time point refers to the start time point of the time period corresponding to the plot summary; if the difference between the minimum time point and the maximum time point is greater than a set time threshold, taking the plot summary corresponding to the time period between the second start time point and the second end time point as the target plot summary.

[0041] The plot summary of the initial video can be processed in segments to obtain multiple plot summaries. The above segmentation method can be segmented according to set time intervals, and accordingly, each plot summary is the plot summary of the video clip corresponding to each time interval. For example: an initial video is 45 minutes, and it can be divided into 9 time intervals according to 5 minutes, and each time interval corresponds to a plot summary. The above segmentation method can also be divided according to the scene conversion situation in the initial video. If a transition occurs in the initial video, the plot summary is segmented at the transition time point, and the plot summary before the transition is used as one plot summary, and the plot summary after the transition is used as another plot summary. It should be noted that the embodiment of the present application only briefly introduces the division method of the plot summary, and is not limited.

[0042] As can be seen from the above, each plot summary is obtained by dividing the overall plot summary of the initial video, so each plot summary has its corresponding time period. The time period is the time period when the corresponding video clip of each plot summary appears in the initial video.

[0043] As shown in Table 2, the time period corresponding to the first plot summary is The time period corresponding to the second plot summary is

[0044] Table 2

[0045]

[0046] If the time period corresponding to the bullet screen keyword is Then the time period corresponding to the bullet screen keyword The time periods corresponding to each plot summary For comparison, if the time period corresponding to the bullet screen keyword Time period corresponding to the plot summary If there is an intersection, the plot summary is used as the target plot summary.

[0047] Furthermore, if the time periods corresponding to multiple plot summaries overlap with the time period corresponding to the bullet screen keyword, the multiple plot summaries are merged in chronological order and used as the target plot summary.

[0048] In a possible implementation, taking one of the plot summaries as an example, a method for determining whether the plot summary is a target plot summary is introduced. Each plot summary can be determined by the following method to determine whether the plot summary is a target plot summary.

[0049] Get the start time point in the time period corresponding to the bullet screen keyword As the first starting time point, get the starting time point in the time period corresponding to the plot summary The first starting time point is used as the second starting time point, and the first starting time point is compared with the second time point, and the maximum value is selected from the two starting time points as the maximum time point.

[0050] Get the end time point in the time period corresponding to the bullet screen keyword As the first end time point, get the end time point in the time period corresponding to the plot summary As the second end time point, the first end time point and the second end time point are compared, and the minimum value is selected from the two end time points as the minimum time point.

[0051] Then, the minimum time point and the maximum time point are subtracted to obtain the difference between the two. If the difference is greater than 0, it indicates that the time period corresponding to the plot summary and the time period corresponding to the barrage theme word have an intersection. If the difference is less than or equal to 0, it indicates that the time period corresponding to the plot summary and the time period corresponding to the barrage theme word do not have an intersection.

[0052] Specifically, formula (1) can be used to determine whether the time period corresponding to a plot summary and the time period corresponding to the bullet screen keyword have an intersection.

[0053]

[0054] in, Indicates the end time point of the time period corresponding to the bullet screen keyword. Indicates the end time point of the time period corresponding to a plot summary. The start time of the time period corresponding to the bullet screen keyword. Indicates the starting time point of the time period corresponding to a plot summary. Represents the minimum value of two end time points, Indicates the maximum value between two start time points.

[0055] If the above formula (1) is greater than 0, that is, the smaller end time point minus the larger start time point is greater than 0, then the time period corresponding to the bullet screen theme word and the time period corresponding to the plot summary partially overlap. If the above formula (1) is less than or equal to 0, that is, the smaller end time point minus the larger start time point is less than or equal to 0, it means that the time period corresponding to the plot summary starts after the time period corresponding to the bullet screen theme word ends, or the time period corresponding to the plot summary ends before the time period corresponding to the bullet screen theme word starts, then the time period corresponding to the bullet screen theme word and the time period corresponding to the plot summary do not overlap.

[0056] In this embodiment, whether the time period corresponding to the barrage theme word and the time period corresponding to the plot summary intersect is determined to determine whether the plot summary is a plot summary related to the barrage theme word, thereby providing a method for determining a target plot summary, making the obtained target plot summary more accurate, thereby improving the accuracy of the plot highlights.

[0057] S103, obtaining a target video segment, where a time period corresponding to the target video segment and a time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target video segment is the time period where the target video segment is located in the initial video.

[0058] The target video clip can be understood as a plot summary related to the plot content displayed by the target video clip and the bullet-screen theme word. In other words, the plot content displayed by the target video clip is the same or similar to the theme content represented by the bullet-screen theme word. Or the theme content represented by the bullet-screen theme word is part of the plot content in the target video clip.

[0059] As described in the above embodiment, the bullet-screen keyword is obtained by aggregating multiple bullet-screen texts within a period of time, and the target video segment is the video segment corresponding to the period of time in the initial video. In other words, the multiple bullet-screen texts that generate the bullet-screen keyword are comments on the plot or characters in the target video segment.

[0060] The bullet-screen keywords are obtained by aggregating bullet-screen text 1, bullet-screen text 2, ..., bullet-screen text N. The subject or topic discussed by bullet-screen text 1, bullet-screen text 2, ..., bullet-screen text N is the same. The i-th bullet-screen text among the N bullet-screen texts is sent the earliest, when the initial video is played to 10:35, and the j-th bullet-screen text among the N bullet-screen texts is sent the latest, when the initial video is played to 15:48. Then the target video segment refers to the video segment played by the initial video when the time period is [10:35, 15:48].

[0061] In a possible implementation, a video segment corresponding to a time period corresponding to a bullet screen keyword is captured from an initial video as a target video segment.

[0062] If the time period corresponding to the bullet screen keyword is Then take the original video The video segment corresponding to the time period is used as the target video segment.

[0063] For example, the time period corresponding to the bullet screen keyword is [10:35, 15:48], that is, from the 10th minute and 35th second to the 15th minute and 48th second. Then the video segment from the 10th minute and 35th second to the 15th minute and 48th second is captured from the initial video as the target video segment.

[0064] In this embodiment, the target video clip corresponding to the barrage topic is captured, so that the pictures obtained subsequently are pictures that are strongly related to the barrage theme words, so that the generated plot highlights are more accurate.

[0065] S104: extract a first picture set from the target video segment, where the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video segment.

[0066] The first picture refers to a picture captured from the initial video and related to the bullet screen keyword. The first picture set includes at least one first picture. The first picture is obtained by extracting frames from the target video segment.

[0067] In a possible implementation, the target video segment is subjected to frame extraction processing according to a video frame extraction method, and the extracted picture can be directly used as the first picture. Alternatively, the extracted picture can be subjected to screening processing such as deduplication and recognition, and the screened picture can be used as the first picture.

[0068] In this embodiment, pictures related to the barrage theme words are extracted through the barrage theme words, so that the extracted pictures are more accurate, so as to further improve the accuracy of the plot highlights.

[0069] S105, generating plot highlights of the target video by combining the bullet screen keywords, target plot summary and target picture collection.

[0070] Plot highlights usually refer to the use of concise and powerful expressions to summarize or emphasize the key points or turning points in the plot. For example, when the plot highlights are describing the relationship between characters, it can be "A and B brothers turn against each other", where "A" and "B" refer to the two characters in the initial video. Another example: when the plot highlights are describing the content of the plot, it can be "C saves the beauty naked, D recovers from fever and wakes up", where "C" and "D" refer to the two characters in the initial video.

[0071] In one possible implementation, the plot highlights of a target video are generated from barrage keywords, target plot summaries, and target images, including: obtaining pre-set prompt words, which are used to instruct a text generation model to output the plot highlights of the target video; generating input information from the prompt words, barrage keywords, target plot summaries, and target image sets, and inputting the input information into the generation model; and receiving the plot highlights of the target video output from the generation model.

[0072] In a possible implementation, a preset prompt is obtained. The prompt is mainly used to obtain a subject word or phrase provided for specific information or answer when interacting with the text generation model. The prompt can help the text generation model understand the user's needs and generate relevant answers or content.

[0073] Specifically, in the process of inputting the barrage topic, the target plot summary and the first picture set into the text generation model, the prompt word corresponding to the barrage can be: "The audience sent a barrage while watching a certain plot: []"; the prompt word corresponding to the target plot summary can be: "The plot is as follows: []". The prompt word corresponding to the first picture set can be: "The screen that the audience may be watching when the barrage is sent is as follows: []". In addition, the prompt word "Please combine the plot content and generate an attractive plot point within a set number of words around the barrage keyword" for the output content of the text generation model can also be input. The above-mentioned set number of words can be set according to the actual situation. For example, the set number of words can be 15.

[0074] Furthermore, the above-mentioned prompt words can be set according to actual conditions, for example: input a prompt word for the output content of the text generation large model "Please combine the plot content and generate a well-balanced plot highlight around the barrage theme words."

[0075] For example, when the bullet-screen keyword is "aaa", the target plot summary is "bbbbb", and the first picture set is "Picture 1, Picture 2, Picture 3", the data input into the text generation model is "The audience sent the bullet-screen when watching a certain plot: [aaa]; the plot is as follows: [bbbbb]; the pictures that the audience may be watching when sending the bullet-screen are as follows: [Picture 1, Picture 2, Picture 3]; please combine the plot content and generate an attractive plot point within 15 words around the bullet-screen topic".

[0076] The text generation model analyzes and processes the input data and outputs a plot highlight. During the generation process, the plot highlight adds visual image information, making the plot highlight content richer and increasing the appeal of the series.

[0077] An embodiment of the present application provides a method for generating plot phrases, the method comprising: determining bullet screen theme information, wherein the bullet screen theme information comprises bullet screen theme words and time periods corresponding to the bullet screen theme words, the bullet screen theme words are phrases determined based on multiple bullet screen texts having an associated relationship, the time periods corresponding to the bullet screen theme words are time periods in which multiple bullet screen texts appear in an initial video, and the initial video is a video watched when a user inputs the bullet screen text; obtaining a target plot summary, wherein the time period corresponding to the target plot summary and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target plot summary is the time period of a video clip corresponding to the target plot summary in the initial video; obtaining a target video clip, wherein the time period corresponding to the target video clip and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target video clip is the time period in which the target video clip is located in the initial video; extracting a first picture set from the target video clip, the first picture set comprising at least one first picture, and the first picture is obtained by extracting frames from the target video clip; and generating plot highlights of the target video using the bullet screen theme words, the target plot summary, and the target picture set.

[0078] In the embodiment of the present application, starting from the bullet screen theme, the pictures in the plot summary and the video clips are analyzed to generate the final plot highlights. Since the bullet screen theme is obtained by analyzing the bullet screen comments input by the user, the generated plot highlights are more in line with the interests of the public, which improves the quality of the plot highlights. In the process of generating the plot highlights, the picture analysis of the video clips is introduced. The picture analysis is visual content, which makes the final plot highlights richer in content and further improves the quality of the plot highlights. In addition, this process avoids the step of manually analyzing the content of the bullet screen, improves the efficiency of generating plot highlights, and avoids the influence of human subjective factors on the plot highlights, further improving the quality of the plot highlights. Therefore, the quality of the plot highlights finally generated by the embodiment of the present application is higher, which can attract more users to watch the video content and improve the dissemination efficiency of film and television works.

[0079] On the basis of the above-mentioned embodiments, the present application further optimizes the method for generating plot highlights, such as Figure 2 As shown, the optimized plot phrase generation method provided in the embodiment of the present application mainly includes:

[0080] S201. Determine the barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words. The barrage theme words are phrases determined based on multiple barrage texts with associated relationships. The time period corresponding to the barrage theme words is a time period when multiple barrage texts appear in an initial video. The initial video is the video watched when the user enters the barrage text.

[0081] S202, obtaining a target plot summary, where the time period corresponding to the target plot summary and the time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video;

[0082] S203, obtaining a target video segment, where a time period corresponding to the target video segment and a time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target video segment is the time period in which the target video segment is located in the initial video.

[0083] Steps S201-S203 provided in the embodiment of the present application are the same as the execution process of S101-S103 provided in the above embodiment. For details, please refer to the description in the above embodiment, and no specific limitations are given in the embodiment of the present application.

[0084] S204: Perform frame extraction processing on the target video segment to obtain a video frame set.

[0085] Frame extraction refers to extracting a series of frames (i.e., a single picture) from a target video segment according to a certain time interval or specific conditions. A video frame set refers to a plurality of video frames obtained after frame extraction from a target video segment.

[0086] A series of single pictures are extracted from the target video clip as video frames according to a certain time interval or specific conditions. Frames are extracted from the target video clip at a rate of 8 frames per second (fps), which means that 8 video frames are extracted from the target video clip every second. Frame extraction can be performed using existing video editing software or existing frame extraction tools, and the frame extraction method is not specifically limited in the embodiments of the present application.

[0087] S205 , deduplicate the video frames in the video frame set to obtain a first picture set, where the first picture set includes a plurality of first pictures whose similarity values ​​are less than a set threshold, and the first picture is a complete video frame.

[0088] The similarity value is an indicator used to evaluate the similarity of the visual content of two or more video frames. The similarity value between video frames refers to the similarity value between any two video frames in the video frame set. For multiple video frames, the similarity value between each two video frames is calculated.

[0089] The similarity between two video frames can be calculated by pixel-level comparison, histogram comparison, feature matching, chord similarity and other methods.

[0090] In one possible implementation, a computer vision algorithm or a deep learning model is used to extract feature vectors of video frames, a distance between feature vectors of two video frames is calculated, and a similarity value of the two video frames is determined based on the distance between the feature vectors of the video frames.

[0091] The similarity threshold may be set according to actual conditions, for example, the similarity threshold may be thres=0.77.

[0092] If the similarity value between two video frames is greater than the set similarity threshold, it indicates that the contents of the two video frames are mostly the same. In order to reduce the amount of subsequent data processing, one of the video frames can be deleted and the other video frame can be retained as the first picture and added to the first picture set.

[0093] In a possible implementation, the specific process of deduplication of video frames is described. Assuming that there are N video frames before deduplication, the size of the similarity matrix S is N×N, and each element S[i][j] in the matrix represents the similarity between the i-th video frame and the j-th video frame. For each pair of video frames, the feature vector of the i-th video frame and the feature vector of the j-th video frame calculate the similarity value, and fill the calculated similarity value in the element S[i][j] position of the similarity matrix S. Furthermore, the similarity can also be normalized to the range of [0, 1].

[0094] The similarity threshold can be set to thres = 0.77, and Deleted_index is the number of the duplicate image, which is used to store the index of all deleted frames. Deleted_index = [] is used to initialize an empty list named Deleted_index. This list will be used to store the deleted column index.

[0095] All row indexes from 0 to N-1 will be traversed. If the current row index r is already in the Deleted_index list, it indicates that the row has been processed and this cycle is skipped. If the current row index r is not in the Deleted_index list, all column indexes from 0 to N-1 will be traversed. If the current column index c is already in the Deleted_index list, it indicates that the column has been processed and this cycle is skipped. If the current column index c is not in the Deleted_index list, then obtain whether the similarity value of the position (r, c) in the similarity matrix S is greater than the preset similarity threshold thres. If the similarity value of the position (r, c) is greater than the preset similarity threshold thres, that is, S[r][c] is greater than thres, then the column index c is added to the Deleted_index list and only the row index r is retained in the similarity matrix.

[0096] In other words, traverse the matrix S and check whether each similarity value exceeds the set similarity threshold. If it exceeds, record the column index of the element and add it to the deleted_index list, which means that the frame pictures corresponding to these columns will be deleted. The above repeated checks on rows and columns ensure that the same index will not be processed multiple times, avoiding duplication of work and maintaining the uniqueness of the index in the list.

[0097] In the embodiment of the present application, the similarity between video frames is used to deduplicate pictures to obtain a first picture set, which can reduce the subsequent data processing amount and improve the data processing speed.

[0098] In one possible implementation, deduplication is performed on video frames in a video frame set to obtain a first picture set, including: deduplication processing is performed on the video frames in the video frame set to obtain a third picture set, the third picture set includes multiple pictures whose similarity values ​​are less than a set threshold; picture recognition is performed on the pictures in the third picture set; if the object name of the object in the recognized picture is the same as the object name in the barrage keyword, the recognized picture is used as the first picture and added to the first picture set.

[0099] The steps of using similarity values ​​to deduplicate video frames can refer to the description in the above embodiment and will not be repeated in the embodiments of the present application.

[0100] After deduplication processing is performed on the video frames in the video frame set to obtain the third picture set, face recognition is performed on each picture in the three picture sets, and the object name of the face in the picture is extracted. In other words, the person in each picture is identified.

[0101] Determine the object name of the face extracted from the image and compare it with the object name in the subject keyword. If the object name of the object in the recognized image is the same as the object name in the barrage keyword, the recognized image is used as the first image and added to the first image set. If the object name of the object in the recognized image is different from the object name in the barrage keyword, the recognized image is not added to the first image set.

[0102] The object name of the face extracted from the picture is "Zhang San", and the object name in the topic keyword is "Zhang San". If the object name of the object in the identified picture is the same as the object name in the bullet-screen topic keyword, the identified picture is taken as the first picture and added to the first picture set. The object name of the object in the identified picture is the same as the object name in the bullet-screen topic keyword, which further determines that the picture displayed in the picture is highly correlated with the topic keyword. The picture and the topic keyword are input into the text generation model together, which can further improve the accuracy of the plot highlights.

[0103] The object name of the face extracted from the picture is "Zhang San", and the object name in the topic keyword is "Zhang Si", so the object name of the object in the identified picture is different from the object name in the barrage keyword, and the identified picture is not added to the first picture set. If the object name of the object in the identified picture is different from the object name in the barrage keyword, it indicates that the picture displayed by the image may be irrelevant to the topic keyword. If the picture and the topic keyword are input into the text generation model together, the amount of data processed by the text generation model will increase, and it may interfere with the analysis of the plot content by the text generation model, resulting in a decrease in the accuracy of improving the plot highlights. Therefore, not inputting the identified picture into the text generation model can reduce the amount of data processed by the large model on the one hand, and avoid the interference of the picture on the large model on the other hand, thereby improving the accuracy of the model analysis.

[0104] S206: Crop the first picture in the first picture set to obtain a second picture set, where the second picture set includes at least one second picture, and the second picture is a local image corresponding to the object area in the first picture.

[0105] The difference between the first picture and the second picture is that the first picture is a complete video frame picture obtained by extracting frames, and the second picture is a picture extracted from the first picture and only includes the face area.

[0106] The key points of each first image are identified using a trained key point recognition network, wherein the key point recognition network includes any one of a Histogram of Oriented Gradients (HOG) network, an Active Appearance Models (AAM), a Multi-task Cascaded Convolutional Networks (MTCNN), and the like.

[0107] In an embodiment of the present application, the key point recognition network recognizes the face region in the first image and the object name corresponding to the face region, and the object name may include a character name. The key point recognition network may include a face recognition network.

[0108] In an embodiment of the present application, after the face recognition network identifies the face area in the first image, it obtains the bounding box corresponding to the face area, and based on the bounding box coordinates, crops the face area from the first image as the second image.

[0109] In one possible implementation, the first picture in the first picture set is cropped to obtain the second picture set, including: cropping the object area in the first picture in the first picture set to obtain a fourth picture set, the fourth picture set including at least one partial image cropped from the object area in the first picture; identifying object key points of the pictures in the fourth picture set; if all the object key points are identified, using the picture with all the object key points identified as the second picture and adding it to the second picture set.

[0110] If multiple face regions are identified in a first image, the bounding boxes corresponding to the respective face regions are obtained, and based on the coordinates of the respective bounding boxes, the multiple face regions are cropped from the first image, and multiple second images are obtained. Since multiple second images are cropped from a first image, it indicates that the plot corresponding to the target video clip is a process of communication between multiple characters, and therefore the second image of each character needs to be extracted.

[0111] Furthermore, after deduplication and deletion of the first pictures that do not contain the face area, there may be multiple first pictures remaining. The face area is intercepted for each first picture to obtain the second picture. In this way, the target object in multiple second pictures may be the same object. For example, the second picture 1 and the second picture 2 are both pictures of character 1.

[0112] At this time, all the intercepted second pictures are classified according to the object category. For example, if all the second pictures have 4 character objects, then all the second pictures are divided into 4 categories, each category being pictures of the same character object.

[0113] Furthermore, after classifying the second picture, if there are many categories of second pictures, that is, there are many characters in the first picture, but many characters are not main characters, and have no impact on the development of the plot, and thus have no impact on the highlight moments of the plot, then this type of second picture can be deleted. Specifically, the name of the target object can be extracted from the barrage theme words, and only the second picture corresponding to the target object name extracted from the barrage theme words is retained, and other second pictures are deleted. For example: if character A and character B are extracted from the barrage theme words, then only the picture of character A and the picture of character B are retained. The pictures corresponding to other characters are deleted.

[0114] In another embodiment, the target object name is extracted from the bullet screen keywords, and the face area corresponding to the target object name is extracted from the first picture as the second picture.

[0115] If multiple second pictures are extracted for the same character, one of the multiple second pictures can be selected as the final second picture. Alternatively, one of the multiple second pictures can be selected as the final second picture according to a set selection strategy.

[0116] The selection strategy may include selecting according to resolution, for example: selecting the second picture with the highest resolution as the final second picture.

[0117] The selection strategy may include selecting according to key points in the face area. Key points refer to coordinate points corresponding to key parts of the face such as the nose, mouth, ears, and eyes. The second image containing the most key points is selected as the final second image.

[0118] Furthermore, the second image can be labeled with the target object name. The purpose of labeling the second image with the target object name is to indicate the target object name corresponding to the second image to the large model, so that the large model can process the data accurately and improve the accuracy of plot cutting.

[0119] S207, generating plot highlights of the target video by combining the bullet screen keywords, the target plot summary, the first picture set and the second picture set.

[0120] Specifically, in the process of inputting the barrage topic, the target plot summary and the first picture set into the text generation model, the prompt word corresponding to the barrage can be: "The audience sent a barrage while watching a certain plot: []"; the prompt word corresponding to the target plot summary can be: "The plot is as follows: []". The prompt word corresponding to the first picture set can be: "The screen that the audience may be watching when the barrage is sent is as follows: []". The prompt word corresponding to the second picture set can be: "The face of the character mentioned in the barrage is: []". In addition, you can also input a prompt word for the output content of the text generation model: "Please combine the plot content and generate an attractive plot point within a set number of words around the barrage theme word". The above-mentioned set number of words can be set according to actual conditions. For example, the set number of words can be 15.

[0121] Furthermore, the above-mentioned prompt words can be set according to actual conditions, for example: input a prompt word for the output content of the text generation large model "Please combine the plot content and generate a well-balanced plot highlight around the barrage theme words."

[0122] For example, when the bullet comment keyword is "aaa", the target plot summary is "bbbbb", the first picture set is "Picture 1, Picture 2, Picture 3", and the second picture is "Picture 4". Then the data input into the text generation model is "The audience sent the bullet comment while watching a certain plot: [aaa]; the plot is as follows: [bbbbb]; the screen that the audience may be watching when sending the bullet comment is as follows: [Picture 1, Picture 2, Picture 3]; the face of the character mentioned in the bullet comment is: [Picture 4]; please combine the plot content and generate an attractive plot point within 15 words around the bullet comment topic".

[0123] The text generation model analyzes and processes the input data and outputs a plot highlight. During the generation process, the plot highlight adds visual image information, making the plot highlight content richer and increasing the appeal of the series.

[0124] Figure 3 Schematic diagram of a plot phrase generating device in an embodiment of the present application. Figure 3 As shown, the plot phrase generation device 30 provided in the embodiment of the present application mainly includes: a barrage determination module 31, a plot summary acquisition module 32, a video clip acquisition module 33, a first picture set extraction module 34 and a plot highlight generation module 35.

[0125] Among them, the barrage determination module 31 is used to determine the barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words, the barrage theme words are phrases determined based on multiple barrage texts with an associated relationship, the time period corresponding to the barrage theme words is the time period when multiple barrage texts appear in the initial video, and the initial video is the video watched when the user inputs the barrage text; the plot summary acquisition module 32 is used to obtain the target plot summary, the time period corresponding to the target plot summary and the time period corresponding to the barrage theme words have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video; the video clip acquisition module 33 is used to obtain the target video clip, the time period corresponding to the target video clip and the time period corresponding to the barrage theme words have an intersection, and the time period corresponding to the target video clip is the time period in which the target video clip is in the initial video; the first picture set extraction module 34 is used to extract a first picture set from the target video clip, the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video clip; the plot highlight generation module 35 is used to generate the plot highlights of the target video by using the barrage theme words, the target plot summary and the target picture set.

[0126] An embodiment of the present application provides a plot highlight generation device, which is used to execute the following process: determine bullet screen theme information, wherein the bullet screen theme information includes bullet screen theme words and time periods corresponding to the bullet screen theme words, the bullet screen theme words are phrases determined based on multiple bullet screen texts with an associated relationship, the time period corresponding to the bullet screen theme words is the time period in which the multiple bullet screen texts appear in an initial video, and the initial video is the video watched when the user inputs the bullet screen text; obtain a target plot summary, the time period corresponding to the target plot summary and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video; obtain a target video clip, the time period corresponding to the target video clip and the time period corresponding to the bullet screen theme words have an intersection, and the time period corresponding to the target video clip is the time period in which the target video clip is located in the initial video; extract a first picture set from the target video clip, the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video clip; generate plot highlights of the target video using the bullet screen theme words, the target plot summary and the target picture set.

[0127] In the embodiment of the present application, the plot highlights start with the barrage theme, analyze the plot summary and the images in the video clips, and generate the final plot highlights. Since the barrage theme is obtained by analyzing the barrage comments input by the user, the generated plot highlights are more in line with the interests of the public, which improves the quality of the plot highlights. In the process of generating the plot highlights, the image analysis of the video clips is introduced. The image analysis is visual content, which makes the final plot highlights richer in content and further improves the quality of the plot highlights. In addition, this process avoids the step of manually analyzing the barrage content, improves the efficiency of generating plot highlights, and avoids the influence of human subjective factors on the plot highlights, further improving the quality of the plot highlights. Therefore, the quality of the plot highlights finally generated by the embodiment of the present application is higher, which can attract more users to watch the video content and improve the dissemination efficiency of film and television works.

[0128] In one possible implementation, the plot summary acquisition module 32 is specifically used to take the minimum value between the first end time point and the second end time point as the minimum time point, wherein the first end time point refers to the end time point of the time period corresponding to the barrage theme word, and the second end time point refers to the end time point of the time period corresponding to the plot summary; take the maximum value between the first start time point and the second start time point as the maximum time point, wherein the first start time point refers to the start time point of the time period corresponding to the barrage theme word, and the second start time point refers to the start time point of the time period corresponding to the plot summary; if the difference between the minimum time point and the maximum time point is greater than the set time threshold, the plot summary corresponding to the time period between the second start time point and the second end time point is used as the target plot summary.

[0129] In one possible implementation, the first picture set extraction module 34 is specifically used to extract frames from the target video segment to obtain a video frame set; deduplicate the video frames in the video frame set to obtain a first picture set, wherein the first picture set includes multiple first pictures whose similarity values ​​are less than a set threshold, and the first picture is a complete video frame.

[0130] In one possible implementation, the first picture set extraction module 34 is specifically used to deduplicate the video frames in the video frame set to obtain a third picture set, wherein the third picture set includes multiple pictures whose similarity values ​​are less than a set threshold; perform picture recognition on the pictures in the third picture set; if the object name of the object in the recognized picture is the same as the object name in the barrage keyword, the recognized picture is used as the first picture and added to the first picture set.

[0131] In a possible implementation, the device also includes: a second picture set acquisition module, which is used to crop the first picture in the first picture set to obtain a second picture set, the second picture set includes at least one second picture, and the second picture is a local image corresponding to the object area in the first picture; a plot highlight generation module 35, which is specifically used to generate the plot highlights of the target video using the barrage keywords, the target plot summary, the first picture set and the second picture set.

[0132] In one possible implementation, the second picture set acquisition module is specifically used to crop the object area in the first picture in the first picture set to obtain a fourth picture set, wherein the fourth picture set includes at least one partial image cropped from the object area in the first picture; perform object key point recognition on the pictures in the fourth picture set; if all object key points are recognized, use the picture with all object key points recognized as the second picture and add it to the second picture set.

[0133] In one possible implementation, the plot highlight generation module 35 is specifically used to obtain pre-set prompt words, which are used to indicate the plot highlights of the target video output by the text generation model; generate input information from the prompt words, barrage keywords, target plot summary and target picture set, and input the input information into the generation model; receive the plot highlights of the target video output from the generation model.

[0134] The plot highlight generation device provided in the embodiment of the present application can execute the plot highlight generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0135] Figure 4 is a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device may include a plot highlight generating device, such as Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of the processor 410 in the electronic device can be one or more. Figure 4 A processor 410 is taken as an example; the processor 410, the memory 420, the input device 430 and the output device 440 in the electronic device can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0136] The memory 420 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the plot highlight generation method in the embodiment of the present invention. The processor 410 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 420, that is, the plot highlight generation method provided in the embodiment of the present invention is implemented.

[0137] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0138] The input device 430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, and may include a keyboard, a mouse, etc. The output device 440 may include a display device such as a display screen.

[0139] This embodiment also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to implement the plot phrase generation method provided in the embodiment of the present invention.

[0140] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention is not limited to the method operations described above, and the computer executable instructions can also execute related operations in the plot highlight generation method provided by any embodiment of the present invention.

[0141] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0142] It is worth noting that in the embodiment of the above-mentioned plot highlight generating device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the various functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0143] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0144] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating plot highlights, characterized in that: The method comprises: Determine barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words, the barrage theme words are phrases determined based on multiple barrage texts having an associated relationship, the time period corresponding to the barrage theme words is a time period in which the multiple barrage texts appear in an initial video, the initial video is a video watched when the user inputs the barrage text, and the initial video is a video watched when the user inputs the barrage text; Obtain a target plot summary, where a time period corresponding to the target plot summary and a time period corresponding to the barrage keyword have an intersection, and the time period corresponding to the target plot summary is a time period of the video clip corresponding to the target plot summary in the initial video; Obtain a target video segment, where a time period corresponding to the target video segment and a time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target video segment is a time period where the target video segment is located in the initial video; Extracting a first picture set from the target video segment, where the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video segment; The bullet screen keywords, the target plot summary and the target picture collection are used to generate the plot highlights of the target video.

2. The method according to claim 1, characterized in that: Get a summary of the target plot, including: The minimum value between the first end time point and the second end time point is taken as the minimum time point, wherein the first end time point refers to the end time point of the time period corresponding to the barrage theme word, and the second end time point refers to the end time point of the time period corresponding to the plot summary; The maximum value between the first start time point and the second start time point is taken as the maximum time point, wherein the first start time point refers to the start time point of the time period corresponding to the barrage theme word, and the second start time point refers to the start time point of the time period corresponding to the plot summary; If the difference between the minimum time point and the maximum time point is greater than a set time threshold, the plot summary corresponding to the time period between the second start time point and the second end time point is used as the target plot summary.

3. The method according to claim 1, characterized in that Extracting a first picture set from the target video segment includes: Performing frame extraction processing on the target video segment to obtain a video frame set; The video frames in the video frame set are deduplicated to obtain a first picture set, wherein the first picture set includes a plurality of first pictures whose similarity values ​​are less than a set threshold, and the first picture is a complete video frame.

4. The method according to claim 3, characterized in that Deduplication of video frames in the video frame set is performed to obtain a first picture set, including: Performing deduplication processing on the video frames in the video frame set to obtain a third picture set, wherein the third picture set includes a plurality of pictures whose similarity values ​​are less than a set threshold; Performing picture recognition on the pictures in the third picture set; If the object name of the object in the recognized picture is the same as the object name in the barrage keyword, the recognized picture is taken as the first picture and added to the first picture set.

5. The method according to claim 1, characterized in that: The method further comprises: Performing cropping processing on a first picture in the first picture set to obtain a second picture set, wherein the second picture set includes at least one second picture, and the second picture is a local image corresponding to an object area in the first picture; The step of generating the plot highlights of the target video by using the bullet screen keyword, the target plot summary and the first picture set includes: The bullet screen keywords, the target plot summary, the first picture set and the second picture set are used to generate plot highlights of the target video.

6. The method according to claim 5, characterized in that Cropping the first picture in the first picture set to obtain a second picture set includes: Performing cropping processing on the object region in the first picture in the first picture set to obtain a fourth picture set, wherein the fourth picture set includes at least one partial image obtained by cropping the object region in the first picture; Performing object key point recognition on the pictures in the fourth picture set; If all the key points of the object are recognized, the picture in which all the key points of the object are recognized is used as the second picture and added to the second picture set.

7. The method according to claim 1, characterized in that The target video plot highlights are generated from the bullet screen keywords, the target plot summary, and the target picture, including: Obtaining a preset prompt word, where the prompt word is used to indicate the plot highlights of the target video output by the text generation model; Generating input information from the prompt word, the bullet screen theme word, the target plot summary and the target picture set, and inputting the input information into the generative model; Receive plot highlights of a target video output from the generative model.

8. A plot highlight generating device, characterized in that: The device comprises: A barrage determination module, used to determine barrage theme information, wherein the barrage theme information includes barrage theme words and time periods corresponding to the barrage theme words, the barrage theme words are phrases determined based on multiple barrage texts with associated relationships, the time periods corresponding to the barrage theme words are time periods in which the multiple barrage texts appear in an initial video, and the initial video is the video watched when the user inputs the barrage text; A plot summary acquisition module is used to acquire a target plot summary, where the time period corresponding to the target plot summary and the time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target plot summary is the time period of the video clip corresponding to the target plot summary in the initial video; A video segment acquisition module is used to acquire a target video segment, wherein the time period corresponding to the target video segment and the time period corresponding to the bullet screen keyword have an intersection, and the time period corresponding to the target video segment is the time period in which the target video segment is located in the initial video; A first picture set extraction module, configured to extract a first picture set from the target video segment, wherein the first picture set includes at least one first picture, and the first picture is obtained by extracting frames from the target video segment; The plot highlight generation module is used to generate the plot highlights of the target video by combining the barrage keywords, the target plot summary and the target picture set.

9. An electronic device, characterized in that: The device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the plot highlight generation method as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for generating plot highlights as described in any one of claims 1 to 7 is implemented.

Images