Video processing method, electronic device and storage medium

Abstract

The embodiments of the present invention provide a video processing method, an electronic device, and a storage medium, which relate to the field of video processing technology. Several bullet screen texts in the bullet screen data of a video are clustered to determine several bullet screen categories. Then, a target bullet screen text is selected from the several bullet screen texts corresponding to each bullet screen category, and the subject information corresponding to each bullet screen category is determined based on the target bullet screen text. Then, based on the bullet screen sending time corresponding to the several bullet screen texts, the bullet screen distribution information of the several bullet screen texts corresponding to each subject information is statistically analyzed in time sequence. Finally, based on the bullet screen distribution information, several video clips that meet the first editing conditions corresponding to each subject information are respectively selected from the video, and the target video clips corresponding to each subject information are generated. Technical problems such as low quality of short video content can be solved. Improve the content quality of short videos to meet the viewing needs of more users.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular to a video processing method, electronic equipment and storage medium. Background Art

[0002] As consumers' demand for fast-paced video viewing grows, it is necessary to edit and process videos with longer playback time into short videos to meet user needs. Currently, the common method of editing short videos from long videos is to manually watch the long videos, select multiple clips from them, and then edit the multiple clips. Since different users determine the highlights of long videos, which is greatly affected by user subjective factors, the content quality of the short videos obtained by the above method cannot be guaranteed and may affect the user's viewing experience. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a video processing method, an electronic device and a storage medium, so as to automatically generate short videos based on the bullet screen data of the videos watched by users, improve the content quality of short videos, and meet the viewing needs of more users. The specific technical solutions are as follows:

[0004] In a first aspect of the present invention, a video processing method is provided, the method comprising:

[0005] Obtaining video bullet screen data, wherein the bullet screen data includes a number of bullet screen texts and the bullet screen sending time corresponding to each bullet screen text;

[0006] Clustering several bullet comment texts to determine several bullet comment categories;

[0007] Selecting a target barrage text from a plurality of barrage texts corresponding to each barrage category, and determining the subject information corresponding to each barrage category according to the target barrage text;

[0008] According to the bullet screen sending time, bullet screen distribution information of a plurality of bullet screen texts corresponding to each theme information is counted in time sequence;

[0009] According to the bullet screen distribution information, several video segments that meet the first editing conditions corresponding to each theme information are respectively selected from the video, and a target video segment corresponding to each theme information is generated.

[0010] Optionally, clustering the plurality of bullet-screen texts to determine a plurality of bullet-screen categories includes:

[0011] Filtering a number of bullet-screen texts to obtain a number of filtered bullet-screen texts;

[0012] Cluster the filtered barrage texts to determine several barrage categories.

[0013] Optionally, clustering the filtered barrage texts to determine several barrage categories includes:

[0014] Calculate the first text similarity between each two bullet-screen texts in the filtered plurality of bullet-screen texts respectively;

[0015] When the first text similarity satisfies a preset similarity condition, two bullet-screen texts corresponding to the first text similarity are respectively used as two nodes, and the two corresponding nodes are associated according to the first text similarity to generate a bullet-screen association graph, and the length of the connecting line is inversely proportional to the first text similarity;

[0016] The bullet comment association graph is clustered to determine a number of bullet comment categories.

[0017] Optionally, the performing graph clustering on the barrage association graph to determine a plurality of barrage categories includes:

[0018] Select any node in the bullet chat association graph as the starting node, walk to the next node according to one of the connection lines connected to the starting node, and determine the corresponding walking probability value;

[0019] Walk to another node according to one of the connection lines connected to the next node, and determine a corresponding second walking probability value;

[0020] Until all nodes in the barrage association graph are wandered, a number of corresponding wandering probability values ​​are determined, and the barrage association value is determined based on the number of wandering probability values;

[0021] When the barrage association value satisfies a preset clustering condition, a plurality of barrage categories are determined according to the barrage association value.

[0022] Optionally, filtering the plurality of bullet-screen texts to obtain the filtered plurality of bullet-screen texts includes:

[0023] Matching a plurality of bullet-screen texts respectively according to preset shielding keywords, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or,

[0024] Matching a plurality of bullet-screen texts respectively according to a preset shielding template, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or,

[0025] According to a preset text classification model, several bullet-screen texts are classified and identified respectively, and the blocked bullet-screen texts are removed to obtain several filtered bullet-screen texts.

[0026] Optionally, selecting a target bullet-screen text from a plurality of bullet-screen texts corresponding to each bullet-screen category includes:

[0027] For a plurality of bullet-screen texts in the same bullet-screen category, respectively calculate the second text similarity between one of the bullet-screen texts and the other bullet-screen texts;

[0028] Weighting the similarities of the plurality of second texts to obtain an evaluation value of one of the bullet comment texts in the bullet comment category;

[0029] A target evaluation value satisfying a target evaluation condition is screened out from a plurality of evaluation values, and a bullet-screen text corresponding to the target evaluation value is used as a target bullet-screen text.

[0030] Optionally, determining the topic information corresponding to each barrage category according to the target barrage text includes:

[0031] The target barrage text is input into a preset topic extraction model for semantic recognition, the corresponding target keyword is determined, and the target keyword is used as the topic information corresponding to the barrage category to which the target barrage text belongs.

[0032] Optionally, the barrage distribution information includes:

[0033] The number of bullet screen texts corresponding to each bullet screen theme in a number of time periods, where the number of time periods is obtained by segmenting the playback sequence;

[0034] or,

[0035] The distribution ratio of several barrage texts corresponding to each barrage theme in several time periods is determined by the number of barrage texts corresponding to each barrage theme and the number of barrage texts corresponding to all barrage texts in each time period.

[0036] Optionally, the selecting, based on the bullet screen distribution information, a plurality of video clips satisfying the first editing conditions corresponding to each theme information from the video respectively includes:

[0037] According to the bullet screen distribution information, determine from the video a number of time periods that meet the first editing conditions corresponding to each theme information;

[0038] Performing camera movement recognition on the video segments corresponding to each time period in the video to determine the segment completeness of the corresponding video segments;

[0039] When the completeness of the segment does not satisfy the second editing condition, expanding the time range of the time period corresponding to the video segment to obtain an updated time period;

[0040] Perform camera movement recognition on the video clips corresponding to the updated time period and determine whether the integrity of the clips meets the second editing condition;

[0041] When the completeness of the segment satisfies the second editing condition, a segment matching the corresponding time period is segmented from the video as a video segment.

[0042] In a second aspect of the present invention, there is also provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0043] Memory, used to store computer programs;

[0044] The processor is used to implement any of the above-mentioned method steps when executing the program stored in the memory.

[0045] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned video processing methods.

[0046] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned video processing methods.

[0047] A video processing method provided by an embodiment of the present invention determines several bullet-screen categories by clustering several bullet-screen texts in the bullet-screen data of a video. Then, a target bullet-screen text is selected from several bullet-screen texts corresponding to each bullet-screen category, and the subject information corresponding to each bullet-screen category is determined based on the target bullet-screen text. Then, based on the bullet-screen sending time corresponding to several bullet-screen texts, the bullet-screen distribution information of several bullet-screen texts corresponding to each subject information is statistically analyzed in time sequence. Finally, based on the bullet-screen distribution information, several video clips that meet the first editing conditions corresponding to each subject information are respectively selected from the video, and target video clips corresponding to each subject information are generated. Among them, the target video clip is a collection of several video clips corresponding to the subject information that the user focuses on when watching the video. This can solve the technical problem that the content quality of the edited short video cannot be guaranteed due to the subjective determination of the wonderful clips of the video, and may affect the user's viewing experience. Thereby improving the content quality of the short video and meeting the viewing needs of more users. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0049] Figure 1 A schematic diagram of a step flow of a video processing method provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the steps of another video processing method provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of a bullet-screen association graph provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of determining multiple bullet chat groups based on a bullet chat association graph provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of a video processing device provided by an embodiment of the present invention;

[0054] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0056] Reference Figure 1 , an embodiment of the present invention provides a video processing method, which may include:

[0057] S101. Obtain the bullet screen data of the video, wherein the bullet screen data includes a number of bullet screen texts and the bullet screen sending time corresponding to each bullet screen text.

[0058] In an embodiment of the present invention, barrage is a user interactive function in a video playback scenario. When a user watches a target video based on his own video client, the barrage input box will be called up at any time to publish the barrage. Among them, the barrage text refers to the text content in the barrage published by the user. After the user sends the barrage, the server can obtain the barrage data corresponding to the video. The barrage data can be understood as the relevant data of all barrages obtained during the playback of the entire video. It may include several barrage texts and the barrage sending time corresponding to each barrage text. Among them, the barrage sending time can be determined based on the playback timing of the video as the time axis, and is consistent with the playback timing of the video.

[0059] S102: Clustering a number of bullet comment texts to determine a number of bullet comment categories.

[0060] In an embodiment of the present invention, after obtaining the bullet screen data of a video, several bullet screen texts in the bullet screen data may be clustered. In one example, the text similarity between two bullet screen texts may be used for clustering, for example, two bullet screen texts with high text similarity may be clustered into one bullet screen category. Thus, after performing the clustering operation, several bullet screen categories are obtained. Among them, each bullet screen category includes several bullet screen texts.

[0061] S103, selecting a target barrage text from a plurality of barrage texts corresponding to each barrage category, and determining the subject information corresponding to each barrage category based on the target barrage text.

[0062] In an embodiment of the present invention, the target barrage text is used to characterize the semantic information of all barrage texts in the barrage category, which can reflect the central idea corresponding to the barrage category. In one example, for several barrage texts in the same barrage category, a barrage text can be randomly selected from several barrage texts as the target barrage text. Therefore, after determining the target barrage text, the topic information corresponding to the corresponding barrage category is determined based on the semantic information of the target barrage text. Among them, the topic information is the target keyword determined according to the semantic information of the target barrage text. It is used to characterize the barrage viewpoints expressed by the barrage texts in each barrage category. In other words, it can also be understood as classifying and marking several barrage categories separately through topic information.

[0063] S104, based on the bullet screen sending time, collect bullet screen distribution information of several bullet screen texts corresponding to each theme information in chronological order.

[0064] In an embodiment of the present invention, after determining the subject information corresponding to several barrage categories, the barrage distribution information of several barrage texts corresponding to each subject information can be statistically calculated in time sequence according to the barrage sending time corresponding to each barrage text in the barrage data. For example, the playback sequence of the video is divided to obtain several time periods. Then, the barrage distribution information can be understood as the distribution of the barrage texts corresponding to each subject information in several time periods. Therefore, according to the barrage distribution information, the time period corresponding to the subject information that the user focuses on can be screened out, and the corresponding video clips can be edited based on the time period.

[0065] S105 . According to the bullet screen distribution information, select a plurality of video clips satisfying the first editing conditions corresponding to each theme information from the video, and generate a target video clip corresponding to each theme information.

[0066] In an embodiment of the present invention, the first editing conditions corresponding to each theme information may be preset, wherein the first editing conditions corresponding to each theme information may be the same or may be different, and no excessive restrictions are made here. For example, the first editing condition may be that the number of bullet screen texts of the corresponding theme information within a certain time period is greater than or equal to a preset number threshold, etc. Therefore, based on the bullet screen distribution information, several video clips that meet the first editing conditions corresponding to each theme information can be screened out from the video. Moreover, the more bullet screen texts sent by the user in the process of watching the video clips that meet the first editing conditions, the more exciting the video content of the corresponding video clip is, or the more popular it is with the user. Therefore, several video clips can be edited together to obtain the target video clips corresponding to each theme information. Thereby improving the content quality of the short video and meeting the viewing needs of more users.

[0067] Reference Figure 2 , an embodiment of the present invention provides another video processing method, which may include:

[0068] S201. Obtain the bullet screen data of the video, wherein the bullet screen data includes a number of bullet screen texts and the bullet screen sending time corresponding to each bullet screen text.

[0069] In the embodiment of the present invention, the description of step S201 refers to the description of step S101.

[0070] For example, for a certain variety show, several barrage texts obtained may be as follows: "Yingying is so good at speaking", "Beibei is so pretty", "Baobei is so cute", "Beibei is so beautiful that she looks good in any photo", "Beibei deserves a beating", "This season is the best, everyone is very decent", "Everyone gets along so well this season", "This season is so harmonious", "This season is really good", "If you don't like it, just say it directly", "If you don't like it, just follow your heart", "I just don't like it", "If you don't like it but still keep hanging on, it's really rubbish", "If you don't like it, just reject it".

[0071] S202: Filter a plurality of bullet-screen texts to obtain a plurality of filtered bullet-screen texts.

[0072] In the embodiment of the present invention, irrelevant bullet texts that generate target video clips can be filtered in advance, which can improve the clustering efficiency and effect of subsequent bullet texts. The semantic information categories of irrelevant bullet texts may include but are not limited to pornography, violence, politics, negative attitudes, etc. For example, filtering can be performed in the following three ways:

[0073] In one example, some shielding keywords can be preset for filtering barrage texts. For example, the shielding keywords can be participles such as "garbage" and "deserve a beating". According to the shielding keywords, several barrage texts are matched respectively. If the corresponding shielding keywords are successfully matched in the barrage text, the successfully matched barrage text is cleared. For example, the above barrage texts "Beibei deserves a beating" and "I don't like it but still hanging on, it's really rubbish" contain shielding keywords respectively. The above two barrage texts are cleared to obtain several filtered barrage texts.

[0074] In another example, some shielding templates can be preset for filtering barrage texts. The shielding templates can be common business templates related to pornography and gambling, such as "offline XXX, contact information XXX". Thus, several barrage texts are matched according to the preset shielding templates. If the corresponding shielding template is successfully matched in the barrage text, the barrage text that is successfully matched is cleared, thereby obtaining several barrage texts after filtering.

[0075] In another example, a text classification model can be preset to classify and identify several barrage texts respectively. The text classification model can be determined by LSTM (Long Short-Term Memory) or BERT (Bidirectional Encoder Representations from Transformer). It can be a semantic recognition based on the semantic information of the barrage text, and output the text category of the corresponding barrage text. For example, the text category can include two types: shielded and non-shielded. If the corresponding barrage text is identified as a shielded type, the shielded barrage text is cleared to obtain several filtered barrage texts. Among them, the text classification model is trained based on sample barrage texts that are manually reviewed and classified.

[0076] Those skilled in the art can select at least one of the above three filtering methods to filter the bullet text according to actual business needs, and no specific limitation is made here. By filtering the bullet text in the above manner, the text quality of the filtered bullet texts can be improved.

[0077] S203, respectively calculating the first text similarity between every two bullet-screen texts in the filtered plurality of bullet-screen texts.

[0078] S204. When the first text similarity satisfies a preset similarity condition, two bullet-screen texts corresponding to the first text similarity are respectively used as two nodes, and the two corresponding nodes are associated according to the first text similarity to generate a bullet-screen association graph.

[0079] S205: performing graph clustering on the bullet comment association graph to determine a number of bullet comment categories.

[0080] In an embodiment of the present invention, the first text similarity between the two barrage texts can be used to preliminarily determine whether the semantic information between the two barrage texts is associated. Among them, similarity conditions can be set in advance, such as the similarity condition that the first text similarity is greater than or equal to the first text threshold, etc. In one example, a first similarity model can be trained using a BERT network, so that every two barrage texts in a number of barrage texts can be input into the first similarity model for similarity calculation, and the first text similarity between every two barrage texts in the filtered barrage texts is calculated respectively, and then it is determined whether the several first text similarities meet the preset similarity conditions.

[0081] Reference Figure 3 As shown, when the first text similarity satisfies the preset similarity condition, the two bullet texts corresponding to the first text similarity are respectively used as two nodes. Figure 3 The first text similarity is used to represent the two corresponding nodes. Figure 3 The length of the connecting line is inversely proportional to the first text similarity. For example, the higher the first text similarity, the shorter the length of the connecting line between the corresponding two nodes; the lower the first text similarity, the shorter the length of the connecting line between the corresponding two nodes.

[0082] After calculating the first text similarity between each two barrage texts in the filtered barrage texts, a barrage association graph can be formed based on a number of nodes and a number of connecting lines, such as Figure 3 As shown. Among them, according to a preset community detection algorithm (i.e., seeking an optimal clustering solution by minimizing entropy), the bullet chat association graph can be clustered to determine a number of bullet chat categories.

[0083] In one example, refer to Figure 4As shown, any node in the barrage association graph can be randomly selected as the starting node, and the next node can be wandered according to one of the connecting lines connected to the starting node, and the corresponding wandering probability value can be determined. Since the length of the connecting line is inversely proportional to the first text similarity, the wandering probability value can be determined according to the first text similarity. For example, the wandering probability value can be the normalized value of the first text similarity corresponding to the connecting line. Walk to another node according to one of the connecting lines connected to the next node, and determine the corresponding second wandering probability value. Until all nodes in the barrage association graph are wandered (the same node can wander multiple times during the wandering process), several corresponding wandering probability values ​​are determined, and the barrage association value is determined based on the several wandering probability values. For example, the barrage association value can be obtained by weighting several wandering probability values. The preset clustering condition can be minimizing the barrage association value. Therefore, the barrage association value that meets the preset clustering condition can be screened out from several barrage association values. And when the barrage association value is minimized, it corresponds to the optimal wandering scheme. As shown in FIG. Figure 4 As shown in the figure, after determining the optimal walking scheme, some potential bullet comment categories will be obtained. For example, by sequentially matching the first text similarity between two nodes, several nodes with high first text similarity are determined as one group of bullet comment categories, and other nodes except the bullet comment category are sequentially matched to determine another group of bullet comment categories. By analogy, several bullet comment categories can be obtained. For example, in Figure 4 Each bullet comment category is represented by a dotted circle. The first text similarity between two bullet comment categories is low, corresponding to Figure 4 The length of the connecting line in is longer. Thus, when the barrage association value meets the preset clustering condition, several barrage categories are determined according to the barrage association value.

[0084] For example, based on the following barrage texts, "Yingying is so good at speaking", "Beibei is so pretty", "Baobei is so cute", "Beibei looks good in any photo", "This season is the best, everyone is very decent", "This season everyone gets along so well", "This season is so harmonious", "This season is really good", "If you don't like it, just say it directly", "If you don't like it, just follow your heart", "I just don't like it", "If you don't like it, just refuse it", three groups of barrage categories can be determined, and the corresponding barrage texts for each group can be as follows:

[0085] [Bullet comment category 1] "Yingying is so good at speaking", "Beibei is so pretty", "Baobei is so cute", "Beibei is so pretty that she looks good in any photo".

[0086] [Bullet comment category 2] "This season is the best, everyone is very decent", "Everyone gets along so well this season", "This season is so harmonious", "This season is really good".

[0087] [Bullet comment category 3] "If you don't like it, just say so", "If you don't like it, just follow your heart", "I just don't like it", "If you don't like it, just reject it."

[0088] S206. Select a target barrage text from a plurality of barrage texts corresponding to each barrage category, and determine the subject information corresponding to each barrage category based on the target barrage text.

[0089] In an embodiment of the present invention, the target barrage text is used to characterize the semantic information of all barrage texts in the barrage category, which can reflect the central idea corresponding to the barrage category. In one example, for several barrage texts in the same barrage category, a barrage text can be randomly selected from several barrage texts as the target barrage text. Therefore, after determining the target barrage text, the topic information corresponding to the corresponding barrage category is determined based on the semantic information of the target barrage text. Among them, the topic information is the target keyword determined according to the semantic information of the target barrage text. It is used to characterize the barrage viewpoints expressed by the barrage texts in each barrage category. In other words, it can also be understood as classifying and marking several barrage categories separately through topic information.

[0090] In another optional embodiment of the invention, in order to improve the accuracy of the subsequent determination of the subject information corresponding to the barrage category, the corresponding target barrage text can be selected based on the text similarity. For several barrage texts in the same barrage category, the second text similarity between one of the barrage texts and other barrage texts is calculated respectively. Several second text similarities are weighted to obtain the evaluation value of one of the barrage texts in the barrage category. Among them, the evaluation value is used to evaluate the accuracy of the barrage text in representing the semantic information embodied by the corresponding barrage category as a whole. For example, the higher the evaluation value, the higher the second text similarity between the corresponding barrage text and other barrage texts, and the closer the barrage text is to the true semantic information of other barrage texts. The target evaluation condition can be the highest evaluation value, and the target evaluation value that meets the target evaluation condition is screened out from several evaluation values, and the barrage text corresponding to the target evaluation value is used as the target barrage text, which can reflect the central semantics corresponding to the barrage category, thereby improving the accuracy of determining the subject information based on the target barrage text.

[0091] For example, for the above [barrage category 1], the second text similarity between the barrage text "Yingying is so good at speaking" and the other three barrage texts is calculated, and after the weighting of the three second text similarities, the corresponding evaluation value is 2.4; the second text similarity between the barrage text "Beibei's good-looking" and the other three barrage texts is calculated, and after the weighting of the three second text similarities, the corresponding evaluation value is 2.95; the second text similarity between the barrage text "How cute Baobei" and the other three barrage texts is calculated, and after the weighting of the three second text similarities, the corresponding evaluation value is 2.8; the second text similarity between the barrage text "Beibei's face value is high, and she looks good in any photo" and the other three barrage texts is calculated, and after the weighting of the three second text similarities, the corresponding evaluation value is 2.75. Among them, the barrage text "Beibei's good-looking" has the highest evaluation value, which is determined as the target evaluation value, and the target barrage text in [barrage category 1] is selected as "Beibei's good-looking".

[0092] Since the number of categories and category names are not clear in the clustering process, the target barrage text can be semantically identified through the topic extraction model to obtain the topic information of each clustering group. Therefore, the topic information can also be understood as the target keyword determined based on the semantic information of the target barrage text. Among them, the topic extraction model can be pre-trained based on models such as BERT, and the corresponding topic information is predetermined. For example, the topic information may include relevant participles such as praising appearance, praising special effects, praising personality, praising plot, etc., and may also include relevant participles corresponding to the points that users generally pay attention to when watching the target video.

[0093] The target barrage text "Beibei's good-looking" is input into the topic extraction model for semantic recognition, and the obtained topic information can be: topic [Appearance] degree [High]. Therefore, the topic information is used to characterize the barrage viewpoints expressed by the barrage texts in each barrage category. In other words, it can also be understood as classifying and labeling several barrage categories separately through topic information. Moreover, in this process, a part of the barrage categories from which no topic information can be extracted can be filtered out. Therefore, in the process of generating short videos, only target video clips with clear topic information are generated, which not only ensures the content uniformity of the target video clips generated by editing several video clips, but also ensures that the target video clips can meet the viewing needs of more users and improve the user's viewing experience.

[0094] S207, based on the barrage sending time, count the barrage distribution information of several barrage texts corresponding to each theme information in chronological order.

[0095] In an embodiment of the present invention, after determining the subject information corresponding to several barrage categories, the barrage distribution information of several barrage texts corresponding to each subject information can be counted in time sequence according to the barrage sending time corresponding to each barrage text in the barrage data. Then, the barrage distribution information can be understood as the distribution of the barrage texts corresponding to each subject information in several time periods. Therefore, according to the barrage distribution information, the time period corresponding to the subject information that the user focuses on can be screened out, and the corresponding video clips can be edited based on the time period.

[0096] In one example, the video playback timing is divided to obtain several time periods. For example, the video playback timing is 0-240s. Correspondingly, the playback timing can be divided according to preset time intervals to obtain several time periods. For example, the preset time interval can be 10S, and the corresponding several time periods can be: 0-10S, 10-20S, ..., 230S-240S. Those skilled in the art can reasonably determine the preset time interval according to the playback timing of the corresponding video, and no excessive restrictions are made here.

[0097] In one example, the barrage distribution information may include the number of barrage texts corresponding to each barrage theme in a number of time periods. For example, the barrage distribution information obtained based on the above-mentioned time periods may be: 0-10s: high appearance

[10] , good special effects [5]; 10-20S: high appearance

[20] , good special effects [1]; ...; 230S-240S: high appearance

[20] , good special effects [3].

[0098] Or, in another example, the distribution ratio of several barrage texts corresponding to each barrage theme in several time periods is determined by the number of barrage texts corresponding to each barrage theme and the number of barrage texts corresponding to all barrage texts in each time period. For example, the barrage distribution information obtained based on the above-mentioned several time periods may be: 0-10s: high appearance [10%], good special effects [5%]; 10-20S: high appearance [20%], good special effects [1%];...; 230S-240S: high appearance [20%], good special effects [3%]. Among them, the distribution ratio may be the ratio between the number of barrage texts corresponding to each barrage theme in the corresponding time period and the number of barrage texts corresponding to all barrage texts in this time period.

[0099] S208. According to the bullet screen distribution information, select a plurality of video segments that meet the first editing conditions corresponding to each theme information from the video, and generate a target video segment corresponding to each theme information.

[0100] In an embodiment of the present invention, the first editing condition corresponding to each theme information may be preset, wherein the first editing condition corresponding to each theme information may be the same or different, and no further limitation is made here. For example, the first editing condition may be that the number of bullet-screen texts of the corresponding theme information within a certain time period is greater than or equal to a preset threshold number, such as the preset threshold number is set to 20. Or the distribution ratio is greater than or equal to a preset ratio threshold, such as the preset ratio threshold is 20%.

[0101] Therefore, based on the barrage distribution information, several video clips that meet the first editing conditions corresponding to each theme information can be screened out from the video. Moreover, the more barrage texts sent by the user in the process of watching the video clips that meet the first editing conditions, the more exciting the video content of the corresponding video clip is, or the more popular it is with users. For example, based on the above barrage distribution information, it can be known that 10-20S: high appearance

[20] and 230S-240S: high appearance

[20] meet the first editing condition corresponding to the theme information [high appearance], so that the video clips of the corresponding time periods (10-20S and 230S-240S) are segmented from the video, and the two video clips are combined to form a target video clip that highlights the relevant characters in the video [high appearance]. Based on the above method, several video clips can be edited together to obtain the target video clips corresponding to each theme information. Thereby improving the content quality of short videos and meeting the viewing needs of more users.

[0102] In an optional embodiment of the invention, in order to prevent the video content in multiple video clips from starting and ending inexplicably. Several time periods that meet the first editing conditions corresponding to each theme information can be determined from the video, such as 10-20S: high appearance [20 items] and 230S-240S: high appearance [20 items] meet the first editing condition corresponding to the theme information [high appearance]. And the video clips corresponding to each time period in the video are subjected to camera movement recognition (scene recognition and shot recognition) to determine the segment integrity of the corresponding video clip. And determine whether the segment integrity meets the second editing condition. Among them, camera movement recognition can use a convolutional neural network, and the segment integrity can be obtained by weighting the scene integrity after scene recognition and the shot integrity after shot recognition.

[0103] In one embodiment, the second editing condition may be that the segment integrity of the video segment within a certain time period is greater than or equal to a preset integrity threshold. For example, the preset integrity threshold is 90%. If the segment integrity does not meet the second editing condition, the time range of the time period corresponding to the video segment is expanded. For example, if the segment integrity at the end of the 10-20S video segment does not meet the second editing condition, the corresponding time period can be expanded from 10-20S to 10-22S, and 10-22S is used as the updated time period. The video segments corresponding to the updated time period are respectively identified by camera movement and whether the segment integrity meets the second editing condition is determined. If the segment integrity does not meet the second editing condition, the time range of the time period corresponding to the video segment is continued to be expanded to update the corresponding time period. The method steps of identifying the camera movement of the video segments corresponding to the updated time period and determining whether the segment integrity meets the second editing condition are repeatedly performed. Only when the segment integrity meets the second editing condition, the segment matching the corresponding time period is segmented from the video as the video segment. For example, if both 10-22S and 230S-240S meet the second editing condition, the videos of 10-22S and 230S-240S are segmented from the video as video segments for generating the target video segment, thereby improving the video quality of the target video segment by ensuring the video integrity of each video segment.

[0104] In an optional embodiment of the invention, the target barrage text corresponding to each subject information can also be determined as the target segment title. On the one hand, the automatic generation of the target segment title can be realized. On the other hand, the target segment title is determined based on the barrage text sent by the user, which can increase the interest of the target video segment, thereby improving the user's viewing experience. After determining the corresponding target segment title, the target video segment corresponding to each subject information is generated based on each segment title and the corresponding several video segments. In this way, short videos are automatically generated based on the barrage data of the user watching the video, wherein the barrage data includes the user's feelings and thoughts after watching the video. The method of automatically generating short videos can improve the content quality of short videos and meet the viewing needs of more users. In addition, the barrage data is data unique to the business party, and it is not easy for other business parties to obtain the corresponding barrage data, which can help the business party produce business content with higher differences based on the barrage data.

[0105] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0106] Reference Figure 5 , an embodiment of the present invention provides a video processing device, which may include:

[0107] The data acquisition module 501 is used to obtain the bullet screen data of the video, and the bullet screen data includes a number of bullet screen texts and the bullet screen sending time corresponding to each bullet screen text.

[0108] The text clustering module 502 is used to cluster a plurality of bullet comment texts and determine a plurality of bullet comment categories.

[0109] The theme determination module 503 is used to select a target barrage text from a plurality of barrage texts corresponding to each barrage category, and determine the theme information corresponding to each barrage category based on the target barrage text.

[0110] The distribution information statistics module 504 is used to count the bullet screen distribution information of several bullet screen texts corresponding to each topic information in time sequence according to the bullet screen sending time.

[0111] The target video segment generating module 505 is used to select a number of video segments that meet the first editing conditions corresponding to each theme information from the video according to the barrage distribution information, and generate a target video segment corresponding to each theme information.

[0112] In an optional embodiment of the invention, the text clustering module 502 may include:

[0113] The text filtering submodule is used to filter several bullet-screen texts to obtain several filtered bullet-screen texts.

[0114] The text clustering submodule is used to cluster the filtered barrage texts and determine several barrage categories.

[0115] In an optional embodiment of the invention, the text clustering submodule may include:

[0116] The first similarity calculation unit is used to respectively calculate the first text similarity between every two bullet-screen texts in the filtered plurality of bullet-screen texts.

[0117] The association graph determination unit is used to, when the first text similarity meets a preset similarity condition, respectively regard the two barrage texts corresponding to the first text similarity as two nodes, and associate the two corresponding nodes according to the first text similarity to generate a barrage association graph, wherein the length of the connecting line is inversely proportional to the first text similarity.

[0118] The graph clustering unit is used to perform graph clustering on the bullet comment association graph to determine a plurality of bullet comment categories.

[0119] In an optional embodiment of the invention, the graph clustering unit may also be used for:

[0120] Any node in the bullet chat association graph is selected as the starting node, and the next node is wandered to according to one of the connection lines connected to the starting node, and the corresponding wandering probability value is determined.

[0121] Walk to another node according to one of the connection lines connected to the next node, and determine a corresponding second walking probability value.

[0122] Until all nodes in the bullet-chat association graph are traversed, a number of corresponding traversal probability values ​​are determined, and the bullet-chat association value is determined based on the number of traversal probability values.

[0123] When the barrage association value satisfies a preset clustering condition, a plurality of barrage categories are determined according to the barrage association value.

[0124] In an optional embodiment of the invention, the text filtering submodule may also be used for:

[0125] Matching a plurality of bullet-screen texts respectively according to preset shielding keywords, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or,

[0126] Matching a plurality of bullet-screen texts respectively according to a preset shielding template, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or,

[0127] According to a preset text classification model, several bullet-screen texts are classified and identified respectively, and the blocked bullet-screen texts are removed to obtain several filtered bullet-screen texts.

[0128] In an optional embodiment of the invention, the theme determination module 503 includes:

[0129] The second similarity calculation submodule is used to calculate the second text similarity between one of the bullet-screen texts and other bullet-screen texts for each of the multiple bullet-screen texts in the same bullet-screen category.

[0130] The evaluation value determination submodule is used to weight the similarities of the plurality of second texts to obtain an evaluation value of one of the bullet comment texts in the bullet comment category.

[0131] The target barrage determination submodule is used to filter out a target evaluation value that meets the target evaluation condition from a plurality of evaluation values, and use the barrage text corresponding to the target evaluation value as the target barrage text.

[0132] In an optional embodiment of the invention, the bullet screen distribution information may include:

[0133] The number of barrage texts corresponding to each barrage theme in a number of time periods, where the number of time periods is obtained by dividing the playback sequence.

[0134] or,

[0135] The distribution ratio of several barrage texts corresponding to each barrage theme in several time periods is determined by the number of barrage texts corresponding to each barrage theme and the number of barrage texts corresponding to all barrage texts in each time period.

[0136] In an optional embodiment of the invention, the target video segment generating module 505 may include:

[0137] The time period determination submodule is used to determine, based on the barrage distribution information, a number of time periods in the video that meet the first editing conditions corresponding to each theme information.

[0138] The completeness determination submodule is used to perform camera movement recognition on the video segments corresponding to each time period in the video to determine the segment completeness of the corresponding video segment.

[0139] The time period updating submodule is used to expand the time range of the time period corresponding to the video segment to obtain an updated time period when the segment completeness does not meet the second editing condition.

[0140] The condition determination submodule is used to perform camera movement recognition on the video segments corresponding to the updated time period and determine whether the segment integrity meets the second editing condition.

[0141] The video segment determination submodule is used to segment the video and extract the segment matching the corresponding time period as the video segment when the segment integrity meets the second editing condition.

[0142] The embodiment of the present invention further provides an electronic device, such as Figure 6As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0143] Memory 603, used for storing computer programs;

[0144] The processor 601 is used to execute the program stored in the memory 603 to implement the following steps:

[0145] Obtaining video bullet screen data, wherein the bullet screen data includes a plurality of bullet screen texts and a bullet screen sending time corresponding to each bullet screen text;

[0146] Clustering several bullet comment texts to determine several bullet comment categories;

[0147] Selecting a target barrage text from a plurality of barrage texts corresponding to each barrage category, and determining the subject information corresponding to each barrage category according to the target barrage text;

[0148] According to the bullet screen sending time, bullet screen distribution information of a plurality of bullet screen texts corresponding to each theme information is counted in time sequence;

[0149] According to the bullet screen distribution information, several video segments that meet the first editing conditions corresponding to each theme information are respectively selected from the video, and a target video segment corresponding to each theme information is generated.

[0150] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0151] The communication interface is used for communication between the above terminal and other devices.

[0152] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0153] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0154] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer executes the video processing method described in any one of the above embodiments.

[0155] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the video processing method described in any one of the above embodiments.

[0156] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0157] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0158] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A video processing method, characterized in that: The method comprises: Obtaining video bullet screen data, wherein the bullet screen data includes a number of bullet screen texts and the bullet screen sending time corresponding to each bullet screen text; Filtering a number of bullet-screen texts to obtain a number of filtered bullet-screen texts; Calculate the first text similarity between each two bullet-screen texts in the filtered plurality of bullet-screen texts respectively; When the first text similarity satisfies a preset similarity condition, two bullet-screen texts corresponding to the first text similarity are respectively used as two nodes, and the two corresponding nodes are associated according to the first text similarity to generate a bullet-screen association graph, and the length of the connecting line is inversely proportional to the first text similarity; Performing graph clustering on the barrage association graph to determine a number of barrage categories; For several barrage texts in the same barrage category, respectively calculate the second text similarities between one barrage text and other barrage texts; weight the several second text similarities to obtain the evaluation value of one barrage text in the barrage category; filter out the target evaluation value that meets the target evaluation condition from the several evaluation values, and use the barrage text corresponding to the target evaluation value as the target barrage text, and determine the topic information corresponding to each barrage category according to the target barrage text; According to the bullet screen sending time, bullet screen distribution information of a plurality of bullet screen texts corresponding to each theme information is counted in time sequence; According to the bullet screen distribution information, several video segments that meet the first editing conditions corresponding to each theme information are respectively selected from the video, and a target video segment corresponding to each theme information is generated.

2. The video processing method according to claim 1, characterized in that: The clustering of the bullet chat association graph is performed to determine a number of bullet chat categories, including: Select any node in the bullet chat association graph as the starting node, walk to the next node according to one of the connection lines connected to the starting node, and determine the corresponding walking probability value; Walk to another node according to one of the connection lines connected to the next node, and determine a corresponding second walking probability value; Until all nodes in the barrage association graph are wandered, a number of corresponding wandering probability values ​​are determined, and the barrage association value is determined based on the number of wandering probability values; When the barrage association value satisfies a preset clustering condition, a plurality of barrage categories are determined according to the barrage association value.

3. The video processing method according to claim 1, characterized in that: The filtering of the plurality of bullet-screen texts to obtain the filtered plurality of bullet-screen texts includes: Matching a plurality of bullet-screen texts respectively according to preset shielding keywords, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or, Matching a plurality of bullet-screen texts respectively according to a preset shielding template, and removing the successfully matched bullet-screen texts to obtain a plurality of filtered bullet-screen texts; and / or, According to a preset text classification model, several bullet-screen texts are classified and identified respectively, and the blocked bullet-screen texts are removed to obtain several filtered bullet-screen texts.

4. The video processing method according to claim 1, characterized in that: Determining the topic information corresponding to each barrage category according to the target barrage text includes: The target barrage text is input into a preset topic extraction model for semantic recognition, the corresponding target keyword is determined, and the target keyword is used as the topic information corresponding to the barrage category to which the target barrage text belongs.

5. The video processing method according to claim 1, characterized in that: The bullet screen distribution information includes: The number of bullet screen texts corresponding to each bullet screen theme in a number of time periods, where the number of time periods is obtained by segmenting the playback sequence; or, The distribution ratio of several barrage texts corresponding to each barrage theme in several time periods is determined by the number of barrage texts corresponding to each barrage theme and the number of barrage texts corresponding to all barrage texts in each time period.

6. The video processing method according to claim 5, characterized in that: The selecting, based on the bullet screen distribution information, a plurality of video clips satisfying the first editing conditions corresponding to the respective theme information from the video respectively comprises: According to the bullet screen distribution information, determine from the video a number of time periods that meet the first editing conditions corresponding to each theme information; Performing camera movement recognition on the video segments corresponding to each time period in the video to determine the segment completeness of the corresponding video segments; When the completeness of the segment does not satisfy the second editing condition, expanding the time range of the time period corresponding to the video segment to obtain an updated time period; Perform camera movement recognition on the video clips corresponding to the updated time period and determine whether the integrity of the clips meets the second editing condition; When the completeness of the segment satisfies the second editing condition, a segment matching the corresponding time period is segmented from the video as a video segment.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images