Video content automatic generation method and system

Abstract

The invention provides a video content automatic generation method and system, and the method comprises the steps: obtaining a hot event and a corresponding original video, and extracting a plurality of event features describing the hot event; querying a plurality of target candidate videos of which the video contents are matched with each event feature from a video library; querying a plurality of target candidate songs of which the lyric contents are matched with each event feature from a song library; respectively combining the target candidate video corresponding to each event feature with the corresponding target candidate song to generate a video clip corresponding to each event feature; and determining a video node corresponding to each event feature in the original video, and inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video. According to the method, the creation efficiency of the short video is improved, and meanwhile, the hot events and content display are combined, so that richer, more interactive and more personalized content experience is provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic video content generation, and in particular to a method and system for automatic video content generation. Background Art

[0002] At present, with the popularization and rapid development of information technology, related applications have gradually been integrated into people's lives, providing people with a variety of services. For example, with the rapid growth of online information and the rapid development of short video content, people have an increasing demand for the combination of hot events and content display.

[0003] Short video platforms provide users with opportunities for independent creation, allowing users to gain attention and recognition through their creative talents. At the same time, the rise of short videos has promoted the diversification of content creation, and everyone can express their creativity and talents through short videos.

[0004] However, the traditional short video production process is cumbersome, involving multiple steps such as shooting, editing, and adding special effects, which requires professional skills and software support. This not only increases the cost of creation, but also prolongs the cycle from conception to release of content, resulting in low efficiency in the creation of short videos.

[0005] In the technical solution with application number CN202210958181.3, although it obtains candidate videos for event sequences, splits the candidate videos into multiple candidate segments, and divides the multiple candidate segments according to their corresponding event nodes to obtain M candidate segment sets corresponding to M event nodes, for each candidate segment set, according to the matching degree between each candidate segment in the candidate segment set and the corresponding event node, at least one candidate segment is determined from the candidate segment set as the target segment of the corresponding event node, and according to the target segments of each of the M event nodes, a video of the event sequence is generated, but the combination of hot events and content display is not realized. Summary of the invention

[0006] The present invention provides a method and system for automatically generating video content, so as to improve the efficiency of creating short videos, and at the same time combine hot events with content display to provide users with a richer, more interactive and personalized content experience.

[0007] To solve the above problems, the present invention adopts the following technical solutions:

[0008] The present invention provides a method for automatically generating video content, comprising:

[0009] Obtaining hot events and original videos corresponding to the hot events;

[0010] Extracting a plurality of event features describing the hot event;

[0011] Querying candidate videos whose video content matches each of the event features from the video library to obtain multiple target candidate videos;

[0012] Searching for candidate songs whose lyrics match each of the event features from a song library to obtain a plurality of target candidate songs;

[0013] After combining the target candidate video corresponding to each of the event features with the corresponding target candidate song, a video clip corresponding to each of the event features is generated;

[0014] Determine the video node corresponding to each event feature in the original video, insert the video clip corresponding to each event feature into the video node corresponding to the original video, and generate the target video.

[0015] Preferably, obtaining hot events includes:

[0016] Obtain user behavior data for each event within a preset time period, wherein the user behavior data includes clicks, shares, comments, and likes;

[0017] Setting the weight corresponding to each type of user behavior data;

[0018] After weighted summing the user behavior data of each event and the corresponding weight, the heat value of each event is obtained;

[0019] All events are sorted in descending order according to the heat values, and events with heat values ​​ranked in the top N are selected as hot events, where N is a positive integer.

[0020] Preferably, obtaining hot events includes:

[0021] Get multiple events within a preset time period;

[0022] Marking each of the events as a network node, and calculating the number of the network nodes to obtain the total number of network nodes;

[0023] For each of the network nodes in the network, respectively calculating the shortest path length from each of the network nodes to other network nodes in the network according to the Dijkstra algorithm;

[0024] After summing up the shortest path lengths from each network node to other network nodes in the network, a target shortest path length of each network node is obtained;

[0025] Calculate the logarithm of the target shortest path length of each of the network nodes with the total number of the network nodes as the base, to obtain a closeness centrality value of each of the network nodes;

[0026] The events corresponding to the network nodes with the largest proximity centrality values ​​are selected as hot events.

[0027] Preferably, candidate videos whose video content matches each of the event features are queried from the video library to obtain multiple target candidate videos, including:

[0028] Obtain video content of each candidate video from the video library;

[0029] Extracting video features of the video content of each candidate video respectively, wherein the video features are used to represent semantic information of the video content of the corresponding candidate video;

[0030] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0031] Vectorizing the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video;

[0032] Calculating the cosine distance between the event vector corresponding to each event feature and the video vectors corresponding to all candidate videos respectively, to obtain the similarity between each event feature and all candidate videos;

[0033] The candidate videos with the greatest similarity corresponding to each of the event features are screened out respectively to obtain multiple target candidate videos.

[0034] Preferably, candidate songs whose lyrics match each of the event features are queried from the song library to obtain a plurality of target candidate songs, including:

[0035] Obtain the lyrics content of each candidate song from the song library;

[0036] Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song;

[0037] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0038] Vectorizing the lyrics features of the lyrics content of each candidate song respectively to obtain a lyrics vector corresponding to each candidate song;

[0039] Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs respectively, to obtain the similarity between each event feature and all candidate songs;

[0040] The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

[0041] Furthermore, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method further includes:

[0042] Extracting all video frames of the target video to obtain multiple first video frames;

[0043] Performing median filtering on each of the first video frames to obtain multiple second video frames;

[0044] According to the video nodes corresponding to each of the second video frames, each of the second video frames is compared with an adjacent previous second video frame to identify multiple pairs of motion vectors;

[0045] According to the multiple pairs of motion vectors and the weighted average algorithm, each second video frame is respectively fused with all previous second video frames to obtain an image sequence, wherein each weight of the weighted average algorithm is inversely proportional to the time distance between each second video frame and the first second video frame;

[0046] The fused image sequence is input into a pre-trained convolutional neural network model to obtain a denoised target video, and the convolutional neural network model is used to restore the noisy image to a clean image.

[0047] Furthermore, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method further includes:

[0048] Filtering out a target event feature representing the main idea of ​​the hot event from the multiple event features;

[0049] Determine the video segment corresponding to the target event feature to obtain the target video segment;

[0050] Convert each video frame of the target video clip into a grayscale image;

[0051] Convolving each of the grayscale images using a Laplacian operator to detect a rapidly changing region in each of the grayscale images;

[0052] Calculating the variance of the fast-changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image.

[0053] The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

[0054] Preferably, converting each video frame of the target video segment into a grayscale image comprises:

[0055] Input each video frame of the target video clip into a pre-built OpenCV library;

[0056] The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

[0057] Furthermore, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method further includes:

[0058] Querying matching transition effects according to each of the event features;

[0059] The corresponding transition effect is added between the original video and the inserted video clip.

[0060] The present invention also provides a video content automatic generation system, comprising:

[0061] An acquisition module, used to acquire hot events and original videos corresponding to the hot events;

[0062] An extraction module, used to extract multiple event features describing the hot event;

[0063] A first query module is used to query candidate videos whose video content matches each of the event features from the video library to obtain multiple target candidate videos;

[0064] The second query module is used to query the song library for candidate songs whose lyrics match each of the event features, and obtain multiple target candidate songs;

[0065] A generation module, used to combine the target candidate video corresponding to each event feature with the corresponding target candidate song, and generate a video clip corresponding to each event feature;

[0066] The insertion module is used to determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0067] Compared with the prior art, the technical solution of the present invention has at least the following advantages:

[0068] The video content automatic generation method and system provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of hot events and matching them with the contents in the video library and the song library, thereby improving the attractiveness of the video and the audience's viewing interest; by combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between video images and music and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce the production cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A flowchart of an embodiment of the method for automatically generating video content of the present invention;

[0070] Figure 2 A flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0071] Figure 3 A flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0072] Figure 4 A flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0073] Figure 5 The present invention is a structural block diagram of an embodiment of the video content automatic generation system of the present invention. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0075] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as S11, S12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0076] It will be understood by those of ordinary skill in the art that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0077] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0078] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0079] Please refer to Figure 1 The present invention provides a method for automatically generating video content, comprising the following steps:

[0080] S11, obtaining hot events and original videos corresponding to the hot events;

[0081] S12, extracting multiple event features describing the hot event;

[0082] S13, searching for candidate videos whose video content matches each of the event features from the video library, to obtain multiple target candidate videos;

[0083] S14, searching the song library for candidate songs whose lyrics match each of the event features, to obtain a plurality of target candidate songs;

[0084] S15, respectively combining the target candidate video corresponding to each of the event features with the corresponding target candidate song to generate a video clip corresponding to each of the event features;

[0085] S16, determining the video node corresponding to each event feature in the original video, and inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0086] This embodiment can use technical means such as news aggregation services and social media monitoring tools to monitor and obtain hot events and related original videos in real time. Then, feature descriptors such as SIFT, SURF or ORB are used to extract key features in the original video, and the key features are used as event features to describe hot events. The event features can describe the core content of the hot events and can be in the form of keywords, sentences or paragraphs, such as keywords that describe the content attributes, dissemination characteristics and social impact of the original video.

[0087] Next, this embodiment can use the video and audio content matching algorithm to find candidate videos that match each event feature from the pre-built video library, and after deleting duplicate candidate videos, finally obtain multiple target candidate videos. Among them, the video library pre-stores candidate videos that are assigned labels, and the labels are used to characterize the video features of the corresponding candidate videos. For example, the label of each candidate video in the video library is obtained, and each event feature is matched with the label of each candidate video to determine the candidate video that matches each event feature.

[0088] Similarly, candidate songs matching each event feature are found from the pre-built song library, and after deleting duplicate candidate songs, multiple target candidate songs are finally obtained. The song library also pre-stores candidate songs with labels assigned to them, and the labels are used to characterize the lyrics features of the corresponding candidate songs.

[0089] In addition, this embodiment combines the target candidate video corresponding to each event feature with the target candidate song corresponding to each event feature to generate a video clip related to the event feature, so as to enhance the emotional expression and visual impact of the video through matching music and video clips. Then, the system determines the corresponding video node of each event feature in the original video, inserts the generated video clip into the corresponding position of the original video, and forms the final target video, so that the generated video is highly relevant to the hot event, can attract the audience's attention, and increase the viewing rate and participation; at the same time, videos closely related to hot events are also easier to spread on social media, optimizing the distribution effect of content. In addition, the automated processing flow also reduces the workload of manual editing and shortens the time of video production.

[0090] For example, suppose there is a hot event called "Music Festival Live" and the original video is a live performance. The system identifies event features such as "music", "crowd", "stage lights" and other event features. Find live performance videos that match "music" and "stage lights" from the video library. Find popular songs that match "Music Festival Live" from the song library, combine the matched video clips with the popular songs, and generate new video clips. In the original video, find the video node that matches each event feature (such as the video node where the music ends, the part where the crowd quiets down, or the video node where the stage lights switch), and insert the newly generated video clip into the corresponding video node of the original video. In the end, a target video that includes both the original live performance and the new live video and music elements is obtained. This video is more vivid and rich, can better attract the audience's attention, and gain more dissemination on social media.

[0091] The video content automatic generation method provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of hot events and matching them with the contents in the video library and the song library, thereby improving the attractiveness of the video and the viewing interest of the audience; by respectively combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between video images and music and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is respectively inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce the production cost.

[0092] In one embodiment, please refer to Figure 2 The step of obtaining hot events may specifically include:

[0093] S111, obtaining user behavior data for each event within a preset time period, wherein the user behavior data includes click volume, share volume, comment volume, and like volume;

[0094] S112, setting a weight corresponding to each type of user behavior data;

[0095] S113, performing weighted summation of the user behavior data of each event and the corresponding weight to obtain a heat value of each event;

[0096] S114. Sort all events in descending order of the heat values, and select the events with the top N heat values ​​as hot events, where N is a positive integer.

[0097] Within a preset time period, such as 7 days, user behavior data for each event is collected from social media, news websites, forums and other platforms through data analysis tools or APIs. The user behavior data may include the number of clicks, shares, comments and likes of the original video, so as to fully reflect the user's attention and participation in different events, and provide basic data for subsequent heat analysis.

[0098] In addition, this embodiment can set the corresponding weights of various types of user behavior data according to their contribution to the event popularity. For example, clicks and shares may better reflect the spread of an event than likes and comments, so they can be given higher weights. By setting weights, the actual impact of different user behaviors on the event popularity can be more accurately reflected, making the calculated popularity value more scientific and reasonable.

[0099] Then, using the weighted summation method, the user behavior data corresponding to each event is multiplied by its corresponding weight and summed up to obtain the heat value of each event, so as to comprehensively consider the importance of different user behaviors and provide a quantitative heat index to help distinguish the popularity of events.

[0100] Finally, according to the calculated heat value, all events are sorted in descending order according to the corresponding heat value, and the events with the top N heat values ​​are selected as hot events. By sorting and screening, it can help quickly identify the most concerned social events.

[0101] For example, suppose you want to identify the top news events of the week. Your system can collect the following user behavior data from news websites and social media platforms:

[0102] Event A: 1,000 clicks, 200 shares, 50 comments, and 300 likes;

[0103] Event B: 800 clicks, 150 shares, 75 comments, and 400 likes.

[0104] The system weights each type of user behavior data as follows:

[0105] Clicks: 0.4;

[0106] Share amount: 0.3;

[0107] Comments: 0.2

[0108] Likes: 0.1.

[0109] Then, the heat value of each event is calculated based on the specific value of each type of user behavior data and the corresponding weight:

[0110] The heat value of event A is: 1000*0.4+200*0.3+50*0.2+300*0.1=500;

[0111] The heat value of event B is: 800*0.4+150*0.3+75*0.2+400*0.1=460.

[0112] Finally, all events are sorted in descending order of popularity. At this point, it can be found that the popularity of event A is higher than that of event B. If N=1 is set, then event A is the hot event of this week. Therefore, hot events can be accurately screened out by quantifying the popularity of events.

[0113] In another embodiment, please refer to Figure 3 The step of obtaining hot events may specifically include:

[0114] S211, acquiring multiple events within a preset time period;

[0115] S212, marking each of the events as a network node, and calculating the number of the network nodes to obtain the total number of network nodes;

[0116] S213. For each of the network nodes in the network, respectively calculate the shortest path length from each of the network nodes to other network nodes in the network according to the Dijkstra algorithm;

[0117] S214, summing the shortest path lengths from each network node to other network nodes in the network to obtain a target shortest path length for each network node;

[0118] S215, calculating the logarithm of the target shortest path length of each of the network nodes with the total number of the network nodes as the base, to obtain a closeness centrality value of each of the network nodes;

[0119] S216: Filter out the events corresponding to the network nodes with the largest proximity centrality values ​​as hot events.

[0120] This embodiment is based on graph theory and network analysis methods to identify hot events from multiple events. Specifically, all events that occurred within a specific time period can be collected through data collection tools, such as news aggregation platforms, social media monitoring tools, etc., and after screening, multiple events within a preset time period are obtained to ensure that the analyzed data is the latest and relevant, which helps to capture hot events with strong timeliness.

[0121] Next, in the mesh topology, each of the events is marked as a network node. Each network node may be connected to two or more network lines. The position of the node is determined by the lines it is connected to and other nodes in the network. At the same time, the total number of network nodes is counted. In graph theory, each event is regarded as a node, and a network model is constructed. By converting events into nodes, the relationship and influence between events can be analyzed using graph theory methods.

[0122] For each node in the network, the Dijkstra algorithm is used to calculate the shortest path length from it to all other nodes. The Dijkstra algorithm can efficiently solve the shortest path problem in a weighted graph and provide the necessary path data for subsequent closeness centrality calculations. Among them, the Dijkstra algorithm is a graph search algorithm that is used to find the shortest path from a single source point to all other nodes in a weighted graph. The Dijkstra algorithm solves the single-source shortest path problem, that is, starting from a specified source point, find the shortest path to all other nodes in the graph.

[0123] For each network node, the shortest path lengths from it to all other network nodes are summed up to obtain the target shortest path length of the node. The target shortest path length can reflect the centrality of a node in the network, that is, the proximity of the node to all other nodes.

[0124] Finally, taking the total number of network nodes as the base, the logarithm of the target shortest path length of each network node is calculated, and the closeness centrality value of each network node is calculated. Based on the calculated closeness centrality value, the network node with the largest closeness centrality value is selected, and the corresponding event is the hot event. The closeness centrality value can quantify the positional importance of the network node in the network. The larger the value, the closer the node is to the network center, so as to objectively identify the most influential hot events.

[0125] For example, suppose the system is analyzing news events within a week to determine which events are hot events. After screening all news events, four news events within a week are collected and marked as four network nodes (A, B, C, D) in the mesh topology. If the Dijkstra algorithm is used to calculate the shortest path length from each network node to all other network nodes, the following is the result:

[0126] The shortest path length from A to B is 1;

[0127] The shortest path length from A to C is 2;

[0128] The shortest path length from A to D is 3;

[0129] The shortest path length from B to C is 1;

[0130] The shortest path length from B to D is 2;

[0131] The shortest path from C to D has a length of 1.

[0132] From the above, we can know that for network node A, its target shortest path length is 1+2+3=6. With 4 as the base, the logarithm of the target shortest path length of network node A is calculated as log46, which is used as the closeness centrality value of network node A. Similarly, the closeness centrality values ​​of the other three network nodes are calculated, and the network node with the largest closeness centrality value is selected. The corresponding event is determined as a hot event, so as to accurately identify the most influential hot event from a large number of events.

[0133] In one embodiment, please refer to Figure 4 The step of searching the video library for candidate videos whose video contents match each of the event features to obtain a plurality of target candidate videos may specifically include:

[0134] S131, obtaining the video content of each candidate video from the video library;

[0135] S132, extracting video features of the video content of each candidate video respectively, where the video features are used to represent semantic information of the video content of the corresponding candidate video;

[0136] S133, performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0137] S134, respectively performing vectorization processing on the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video;

[0138] S135, respectively calculating the cosine distance between the event vector corresponding to each of the event features and the video vectors corresponding to all the candidate videos, to obtain the similarity between each of the event features and all the candidate videos;

[0139] S136 , respectively select the candidate videos with the greatest similarity corresponding to each of the event features to obtain multiple target candidate videos.

[0140] In this embodiment, each candidate video is obtained from a pre-built video library, and the video content of each candidate video is extracted. The video features of the video content of each candidate video are extracted using a video analysis method (such as a 3D convolutional neural network CNN). The video features can characterize the semantic information of the video content, such as describing the video type (entertainment, sports, education, etc.) of the video content, the scene type in the video (such as indoor, outdoor, specific location, etc.), or the object in the video (such as a person, vehicle, animal, etc.) in the form of keywords, sentences, or text paragraphs. The video feature extraction can help understand the video content of the candidate video and provide key information for the subsequent video similarity calculation.

[0141] Secondly, each event feature is vectorized by one-hot encoding or calling the Word2Vec model to convert text, audio or visual features into numerical vectors, namely event vectors, and the extracted video features of each candidate video are vectorized to obtain video vectors. This vectorization process allows event features and video features to be used for mathematical calculations and comparisons, facilitating subsequent similarity calculations.

[0142] Among them, One-Hot Encoding, also known as One-Efficient Encoding, is a method of converting categorical variables into a form that can be better processed by machine learning algorithms. In One-Hot Encoding, each category value is represented as a binary vector, except for one position representing the category, which is 1, and the rest of the positions are 0.

[0143] The Word2Vec model is a model for generating word embeddings. The Word2Vec model generates word vectors by learning the contextual relationship between words, which can capture the semantic and syntactic information of words. The output of the Word2Vec model is a set of vectors, each word corresponds to a vector of fixed dimension, which can be used for various natural language processing (NLP) tasks.

[0144] Then, this embodiment can use the cosine distance formula to calculate the cosine distance between each event vector and all candidate video vectors, and the cosine distance is used to characterize the similarity between each event feature and each corresponding candidate video. The cosine distance measures the angle between two vectors. The smaller the angle, the higher the similarity. Finally, according to the calculated similarity, the candidate video with the largest similarity corresponding to each event feature is screened out. The screened candidate video is the most matched with the event feature, which improves the accuracy and relevance of video retrieval.

[0145] For example, suppose you are making a short video about "city marathon" and need to find the most relevant video clips from the video library. First, get all the videos marked as "marathon" from the video library, use the deep learning model to extract the visual features of these videos, such as scenes, character movements, etc., vectorize the event features of "city marathon" (such as urban landscape, running crowd), and vectorize the video features at the same time, calculate the cosine distance between the "city marathon" event vector and each candidate video vector, and select the video with the smallest cosine distance. This video is most similar to the "city marathon" event features, so as to quickly filter out the most relevant video clips from a large number of videos and provide powerful visual materials for short video production.

[0146] In one embodiment, the step of searching the song library for candidate songs whose lyrics match each of the event features to obtain a plurality of target candidate songs may specifically include:

[0147] Obtain the lyrics content of each candidate song from the song library;

[0148] Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song;

[0149] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0150] Vectorizing the lyrics features of the lyrics content of each candidate song respectively to obtain a lyrics vector corresponding to each candidate song;

[0151] Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs respectively, to obtain the similarity between each event feature and all candidate songs;

[0152] The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

[0153] This embodiment extracts each candidate song from a pre-built song library, extracts the lyrics content of each candidate song, and uses natural language processing (NLP) to extract semantic features of the lyrics content, such as theme, emotion, keywords, etc. This lyrics feature extraction helps to understand the deep meaning of the lyrics and provides key information for subsequent similarity calculations.

[0154] Secondly, each event feature is vectorized by one-hot encoding or calling the Word2Vec model, and the text, audio or visual features are converted into numerical vectors, namely event vectors, and the lyrics features of each candidate song are vectorized to obtain lyrics vectors. This vectorization process allows event features to be used for mathematical calculations and comparisons, facilitating subsequent similarity calculations.

[0155] Among them, One-Hot Encoding, also known as One-Efficient Encoding, is a method of converting categorical variables into a form that can be better processed by machine learning algorithms. In One-Hot Encoding, each category value is represented as a binary vector, except for one position representing the category, which is 1, and the rest of the positions are 0.

[0156] The Word2Vec model is a model for generating word embeddings. The Word2Vec model generates word vectors by learning the contextual relationship between words, which can capture the semantic and syntactic information of words. The output of the Word2Vec model is a set of vectors, each word corresponds to a vector of fixed dimension, which can be used for various natural language processing (NLP) tasks.

[0157] Then, this embodiment uses the cosine distance formula to calculate the cosine distance between each event vector and the lyrics vectors of all candidate songs. The cosine distance is used to characterize the similarity between each event feature and each corresponding candidate song. The cosine distance measures the angle between two vectors. The smaller the angle, the higher the similarity. Finally, based on the calculated similarity, the candidate songs with the greatest similarity to each event feature are screened out. The screened candidate songs are most compatible with the event features, which improves the accuracy and relevance of song retrieval.

[0158] For example, suppose you are making a special program about "graduation season" and need to find the most relevant lyrics from the song library. First, get the lyrics of all candidate songs related to themes such as "graduation", "youth", and "farewell" from the song library. Then use NLP technology to extract the theme, emotion, keywords and other features of these lyrics. Vectorize the event features of "graduation season" (such as graduation, youth, and farewell), and vectorize the lyrics features at the same time, calculate the cosine distance between the "graduation season" event vector and the lyrics vector of each candidate song, and select the songs with the smallest cosine distance. The lyrics of these songs are most similar to the "graduation season" event features, so as to quickly filter out the lyrics most relevant to the theme of "graduation season" from a large number of songs, and provide powerful music materials for short video production.

[0159] In yet another embodiment, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method may further include:

[0160] Extracting all video frames of the target video to obtain multiple first video frames;

[0161] Performing median filtering on each of the first video frames to obtain multiple second video frames;

[0162] According to the video nodes corresponding to each of the second video frames, each of the second video frames is compared with an adjacent previous second video frame to identify multiple pairs of motion vectors;

[0163] According to the multiple pairs of motion vectors and the weighted average algorithm, each second video frame is respectively fused with all previous second video frames to obtain an image sequence, wherein each weight of the weighted average algorithm is inversely proportional to the time distance between each second video frame and the first second video frame;

[0164] The fused image sequence is input into a pre-trained convolutional neural network model to obtain a denoised target video, and the convolutional neural network model is used to restore the noisy image to a clean image.

[0165] This embodiment describes a video processing flow, which aims to improve the quality of a target video through median filtering, motion vector recognition, image fusion, and deep learning noise reduction.

[0166] Specifically, firstly, images are extracted frame by frame from the target video to form a series of independent video frames to obtain multiple first video frames, and each extracted first video frame is processed by applying a median filter to obtain multiple second video frames, so as to remove noise of the first video frame, especially salt and pepper noise, while maintaining edge information of the first video frame, thereby improving the image quality of the first video frame.

[0167] Secondly, the optical flow algorithm is used to compare the two adjacent second video frames that have been processed by the median filter to identify the motion vector between them, that is, the position change of the pixel between the two frames. The identification of this motion vector helps to understand the motion information in the video and provide a basis for subsequent image fusion.

[0168] Among them, the optical flow algorithm is an algorithm used in the field of computer vision to estimate the motion pattern of pixels in an image sequence. It is based on the principle of constant brightness, that is, between consecutive image frames, the brightness of the pixels remains unchanged, and the speed and direction of the pixel movement are consistent. With this information, the optical flow algorithm can construct a vector field, each vector representing the speed and direction of the object point in the image, that is, the motion vector.

[0169] In one embodiment, each second video frame is compared with the adjacent previous second video frame, and when multiple pairs of motion vectors are identified, each second video frame can be aligned with the adjacent previous second video frame, and then the corner points in each second video frame can be identified according to a corner point detection algorithm, such as Harris corner point detection or Shi-Tomasi corner point detection, and the corner points whose brightness change from the previous second video frame to the next second video frame is greater than a threshold are selected as feature points. The corner point detection algorithm can find an area with obvious brightness change, which is more likely to correspond to the same feature point on different frames of the same object in the image sequence.

[0170] For each detected feature point, an optical flow equation is established. The optical flow equation is based on the assumption of constant brightness, that is, the brightness of the same object remains unchanged in consecutive image frames. For each feature point, the optical flow vector is estimated by minimizing the weighted sum of squares of the residual according to the optical flow equation, thereby identifying multiple pairs of motion vectors. The residual is the difference between the actual brightness value and the brightness value predicted according to the optical flow assumption.

[0171] Then, this embodiment can use a weighted average algorithm to fuse each processed second video frame with all the previous second video frames according to the motion vector to obtain an image sequence. Specifically, the motion vector is used to align the current second video frame with the adjacent previous second video frame, and the alignment process involves adjusting the pixel position according to the motion vector. After alignment, the weighted average algorithm is used to fuse the current second video frame with the adjacent previous second video frame, and the fused video frame is then fused with all the previous second video frames to finally obtain an image sequence, in which each frame of the image sequence contains the information of all the previous frames, thereby enhancing the continuity and integrity of the video content.

[0172] Among them, each weight of the weighted average algorithm is inversely proportional to the corresponding time distance, and the time distance is the time interval between each second video frame and the first second video frame. It should be noted that when the second video frame is the first second video frame, the time distance of the second video frame is 0, and the corresponding weight is also 0. This step can reduce the flicker and discontinuity between frames through image fusion, and improve the stability and fluency of the video.

[0173] Finally, the fused image sequence is input into a pre-trained convolutional neural network model for denoising. The convolutional neural network model can learn complex image features and effectively restore clean images from noisy images.

[0174] This embodiment can significantly improve the quality of low-light videos, reduce noise, and make videos clearer and smoother through image processing methods such as median filtering, motion vector recognition, image fusion, and deep learning noise reduction.

[0175] In one embodiment, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the following steps may also be included:

[0176] Filtering out a target event feature representing the main idea of ​​the hot event from the multiple event features;

[0177] Determine the video segment corresponding to the target event feature to obtain the target video segment;

[0178] Convert each video frame of the target video clip into a grayscale image;

[0179] Convolving each of the grayscale images using a Laplacian operator to detect a rapidly changing region in each of the grayscale images;

[0180] Calculating the variance of the fast-changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image.

[0181] The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

[0182] This embodiment aims to filter out the features that best represent the hot events from multiple event features, and select the video cover accordingly. Specifically, firstly, multiple event features are analyzed to identify the core features that best represent the hot events, so that the hot events can be more accurately located by filtering out the main features, providing key information for subsequent video processing and cover selection.

[0183] Next, according to the selected target event features, the corresponding video clips are extracted from the video as target video clips to ensure that the video clips are closely related to the hot events, thereby improving the pertinence and relevance of the video content.

[0184] Then, each frame in the video clip is converted into a grayscale image to simplify the data and highlight the structural information of the image. Grayscale image processing is simpler and less computationally intensive than color image processing, while retaining important image features such as edges and textures.

[0185] Apply the Laplacian operator to each grayscale image to detect fast-changing areas in the image, i.e., edges. The Laplacian operator can highlight the edge information in the image, which helps with subsequent image clarity assessment.

[0186] In addition, this embodiment calculates the variance of the fast-changing area in the image processed by the Laplacian operator as a measure of image clarity. The larger the variance, the greater the local contrast of the image, the clearer the image, which helps to select the most representative video frame as the cover. Finally, based on the calculated clarity, the video frame corresponding to the grayscale image with the highest clarity is selected as the video cover to attract the audience's attention and increase the click-through rate and viewing rate of the video.

[0187] This embodiment can ensure that the video cover has high definition and can accurately reflect the core content of the hot event, thereby improving the attractiveness and dissemination effect of the video.

[0188] In one embodiment, converting each video frame of the target video segment into a grayscale image includes:

[0189] Input each video frame of the target video clip into a pre-built OpenCV library;

[0190] The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

[0191] This embodiment involves using the OpenCV library to grayscale the video frames of the target video clip. Specifically, each video frame is read from the target video clip frame by frame, and is passed to the OpenCV library as input, and each video frame is converted from a color image to a grayscale image using the cvtColor() function in the OpenCV library. The grayscale processing simplifies the video frames of the target video clip, reduces the computational complexity, and retains the structural information of the video frames.

[0192] Among them, OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. The OpenCV library contains hundreds of computer vision algorithms that can be used to process and analyze image and video data.

[0193] The cvtColor() function is a function in the OpenCV library that is used to convert the color space of an image. The cvtColor() function can change an image from one color space to another, such as from BGR (blue green red) to RGB (red green blue), or from a color image to a grayscale image.

[0194] In one embodiment, after inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the following steps may also be included:

[0195] Querying matching transition effects according to each of the event features;

[0196] The corresponding transition effect is added between the original video and the inserted video clip.

[0197] This embodiment can match and add transition effects according to event features during the video production process to enhance the visual effect and narrative fluency of the video.

[0198] Specifically, first, analyze the video content and event characteristics, and determine the appropriate transition effects to ensure that the transition effects match the emotion and rhythm of the video content, thereby improving the professionalism and viewing experience of the video.

[0199] Secondly, this embodiment can use video editing software or programming libraries (such as OpenCV) to add selected transition effects between the original video and the inserted video clip. The transition effects can smoothly connect the original video and the video clip, reduce visual jumps, and enhance the coherence and narrative fluency of the video.

[0200] For example, suppose you are editing an original video about a "city marathon" and need to add transition effects between different segments. Then, based on the event features of the "city marathon", such as "start", "mid-race", and "finish", query the database to match suitable transition effects. For example, "start" may match the transition effect of "fast blur to clear", while "finish" may match the transition effect of "slow motion to normal speed". Then, using OpenCV or a similar video processing library, write code to add the selected transition effect between the corresponding segments of the original video to reduce visual jumps and enhance the coherence and narrative fluency of the video.

[0201] Please refer to Figure 5 The embodiment of the present invention further provides a video content automatic generation system, comprising:

[0202] An acquisition module 51 is used to acquire hot events and original videos corresponding to the hot events;

[0203] An extraction module 52, used to extract multiple event features describing the hot event;

[0204] A first query module 53 is used to query the video library for candidate videos whose video content matches each of the event features, and obtain multiple target candidate videos;

[0205] The second query module 54 is used to query the song library for candidate songs whose lyrics match each of the event features, and obtain multiple target candidate songs;

[0206] A generating module 55, for combining the target candidate video corresponding to each event feature with the corresponding target candidate song, to generate a video clip corresponding to each event feature;

[0207] The insertion module 56 is used to determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0208] The video content automatic generation system provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of hot events and matching them with the contents in the video library and the song library, thereby improving the attractiveness of the video and the viewing interest of the audience; by respectively combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between video images and music and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is respectively inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce the production cost.

[0209] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0210] In one embodiment, the present invention further proposes a storage medium storing computer-readable instructions, which, when executed by one or more processors, causes one or more processors to execute the above-mentioned video content automatic generation method. The storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0211] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0212] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for automatically generating video content, characterized in that: include: Obtaining hot events and original videos corresponding to the hot events; Extracting a plurality of event features describing the hot event; Querying candidate videos whose video content matches each of the event features from the video library to obtain multiple target candidate videos; Searching for candidate songs whose lyrics match each of the event features from a song library to obtain a plurality of target candidate songs; After combining the target candidate video corresponding to each of the event features with the corresponding target candidate song, a video clip corresponding to each of the event features is generated; Determine the video node corresponding to each event feature in the original video, insert the video clip corresponding to each event feature into the video node corresponding to the original video, and generate the target video.

2. The method for automatically generating video content according to claim 1, characterized in that: Get hot events, including: Obtain user behavior data for each event within a preset time period, wherein the user behavior data includes clicks, shares, comments, and likes; Setting the weight corresponding to each type of user behavior data; After weighted summing the user behavior data of each event and the corresponding weight, the heat value of each event is obtained; All events are sorted in descending order according to the heat values, and events with heat values ​​ranked in the top N are selected as hot events, where N is a positive integer.

3. The method for automatically generating video content according to claim 1, characterized in that: Get hot events, including: Get multiple events within a preset time period; Marking each of the events as a network node, and calculating the number of the network nodes to obtain the total number of network nodes; For each of the network nodes in the network, respectively calculating the shortest path length from each of the network nodes to other network nodes in the network according to the Dijkstra algorithm; After summing up the shortest path lengths from each network node to other network nodes in the network, a target shortest path length of each network node is obtained; Calculate the logarithm of the target shortest path length of each of the network nodes with the total number of the network nodes as the base, and obtain the closeness centrality value of each of the network nodes; The events corresponding to the network nodes with the largest proximity centrality values ​​are selected as hot events.

4. The method for automatically generating video content according to claim 1, characterized in that: The candidate videos whose video contents match the characteristics of each event are searched from the video library to obtain multiple target candidate videos, including: Obtain video content of each candidate video from the video library; Extracting video features of the video content of each candidate video respectively, wherein the video features are used to represent semantic information of the video content of the corresponding candidate video; Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features; Vectorizing the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video; Calculating the cosine distance between the event vector corresponding to each event feature and the video vectors corresponding to all candidate videos respectively, to obtain the similarity between each event feature and all candidate videos; The candidate videos with the greatest similarity corresponding to each of the event features are screened out respectively to obtain multiple target candidate videos.

5. The method for automatically generating video content according to claim 1, characterized in that: The candidate songs whose lyrics match the event characteristics are searched from the song library to obtain multiple target candidate songs, including: Obtain the lyrics content of each candidate song from the song library; Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song; Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features; Vectorizing the lyrics features of the lyrics content of each candidate song respectively to obtain a lyrics vector corresponding to each candidate song; Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs respectively, to obtain the similarity between each event feature and all candidate songs; The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

6. The method for automatically generating video content according to claim 1, characterized in that: Inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video further includes: Extracting all video frames of the target video to obtain multiple first video frames; Performing median filtering on each of the first video frames to obtain multiple second video frames; According to the video nodes corresponding to each of the second video frames, each of the second video frames is compared with an adjacent previous second video frame to identify multiple pairs of motion vectors; According to the multiple pairs of motion vectors and the weighted average algorithm, each second video frame is respectively fused with all previous second video frames to obtain an image sequence, wherein each weight of the weighted average algorithm is inversely proportional to the time distance between each second video frame and the first second video frame; The fused image sequence is input into a pre-trained convolutional neural network model to obtain a denoised target video, and the convolutional neural network model is used to restore the noisy image to a clean image.

7. The method for automatically generating video content according to claim 1, characterized in that: After inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method further includes: Filtering out a target event feature representing the main idea of ​​the hot event from the multiple event features; Determine the video segment corresponding to the target event feature to obtain the target video segment; Convert each video frame of the target video clip into a grayscale image; Convolving each of the grayscale images using a Laplacian operator to detect a rapidly changing region in each of the grayscale images; Calculating the variance of the fast-changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image. The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

8. The method for automatically generating video content according to claim 7, characterized in that: Convert each video frame of the target video clip into a grayscale image, including: Input each video frame of the target video clip into a pre-built OpenCV library; The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

9. The method for automatically generating video content according to claim 1, characterized in that: After inserting the video clips corresponding to each of the event features into the video nodes corresponding to the original video to generate the target video, the method further includes: Querying matching transition effects according to each of the event features; The corresponding transition effect is added between the original video and the inserted video clip.

10. A video content automatic generation system, characterized in that: include: An acquisition module, used to acquire hot events and original videos corresponding to the hot events; An extraction module, used for extracting a plurality of event features describing the hot event; A first query module is used to query candidate videos whose video content matches each of the event features from the video library to obtain multiple target candidate videos; A second query module is used to query candidate songs whose lyrics match each of the event features from the song library to obtain multiple target candidate songs; A generation module, used to combine the target candidate video corresponding to each event feature with the corresponding target candidate song, and generate a video clip corresponding to each event feature; The insertion module is used to determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

Images