Method and system for automatically generating video content

Abstract

The present invention provides a method and system for automatically generating video content. The method comprises the following steps: obtaining a hot event and its corresponding original video, extracting multiple event features describing the hot event; querying a plurality of target candidate videos whose video content matches each event feature from a video library; querying a plurality of target candidate songs whose lyrics match each event feature from a song library; combining the target candidate videos corresponding to each event feature with the corresponding target candidate songs to generate a video clip corresponding to each event feature; determining the video node corresponding to each event feature in the original video, and inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video. The present invention improves the efficiency of short video creation and combines hot events with content display to provide users with a richer, more interactive, and personalized content experience.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic video content generation, and in particular to a method and system for automatic video content generation. Background Art

[0002] Currently, with the popularization and rapid development of information technology, related applications have gradually been integrated into people's lives, providing people with a variety of services. For example, with the rapid growth of online information and the rapid development of short video content, people are increasingly demanding the integration of hot events and content display.

[0003] Short video platforms provide users with the opportunity to create their own content, allowing them to gain attention and recognition through their creative talents. At the same time, the rise of short videos has promoted the diversification of content creation, allowing everyone to express their creativity and talents through short videos.

[0004] However, the traditional short video production process is cumbersome, involving multiple steps such as filming, editing, and adding special effects, requiring specialized skills and software. This not only increases production costs but also prolongs the cycle from conception to release, resulting in low efficiency in short video creation.

[0005] In the technical solution with application number CN202210958181.3, although it obtains candidate videos for event sequences, splits the candidate videos into multiple candidate segments, and divides the multiple candidate segments according to their corresponding event nodes to obtain M candidate segment sets corresponding to M event nodes, for each candidate segment set, according to the matching degree between each candidate segment in the candidate segment set and the corresponding event node, determines at least one candidate segment from the candidate segment set as the target segment of the corresponding event node, and generates a video of the event sequence according to the target segments of each of the M event nodes, it does not achieve the combination of hot events and content display. Summary of the Invention

[0006] The present invention provides a method and system for automatically generating video content to improve the efficiency of short video creation, while combining hot events with content display to provide users with a richer, more interactive and personalized content experience.

[0007] In order to solve the above problems, the present invention adopts the following technical solutions:

[0008] The present invention provides a method for automatically generating video content, comprising:

[0009] Obtaining hot events and original videos corresponding to the hot events;

[0010] Extracting multiple event features describing the hot event;

[0011] Querying a video library for candidate videos whose video content matches each of the event features to obtain multiple target candidate videos;

[0012] Searching for candidate songs whose lyrics match each of the event features from a song library to obtain a plurality of target candidate songs;

[0013] After combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated;

[0014] Determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0015] Preferably, obtaining hot events includes:

[0016] Obtain user behavior data for each event within a preset time period, including clicks, shares, comments, and likes;

[0017] Setting the weight corresponding to each type of user behavior data;

[0018] After weighted summing the user behavior data of each event and the corresponding weight, the heat value of each event is obtained;

[0019] All events are sorted in descending order of the heat values, and events with heat values ​​in the top N are selected as hot events, where N is a positive integer.

[0020] Preferably, obtaining hot events includes:

[0021] Get multiple events within a preset time period;

[0022] Marking each of the events as a network node, and calculating the number of the network nodes to obtain the total number of network nodes;

[0023] For each of the network nodes in the network, respectively calculating the shortest path length from each of the network nodes to other network nodes in the network according to the Dijkstra algorithm;

[0024] After summing the shortest path lengths from each network node to other network nodes in the network, a target shortest path length for each network node is obtained;

[0025] Calculating the logarithm of the target shortest path length of each of the network nodes with the total number of the network nodes as the base, to obtain a closeness centrality value of each of the network nodes;

[0026] The events corresponding to the network nodes with the largest proximity centrality values ​​are selected as hot events.

[0027] Preferably, candidate videos whose video content matches each of the event features are searched from the video library to obtain multiple target candidate videos, including:

[0028] Obtain the video content of each candidate video from the video library;

[0029] Extracting video features of the video content of each candidate video respectively, wherein the video features are used to represent semantic information of the video content of the corresponding candidate video;

[0030] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0031] Performing vectorization processing on the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video;

[0032] Calculating the cosine distance between the event vector corresponding to each event feature and the video vectors corresponding to all candidate videos, respectively, to obtain the similarity between each event feature and all candidate videos;

[0033] The candidate videos with the greatest similarity corresponding to each of the event features are screened out respectively to obtain multiple target candidate videos.

[0034] Preferably, candidate songs whose lyrics match each of the event features are searched from the song library to obtain a plurality of target candidate songs, including:

[0035] Obtain the lyrics of each candidate song from the song library;

[0036] Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song;

[0037] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0038] Vectorizing the lyrics features of the lyrics content of each candidate song to obtain a lyrics vector corresponding to each candidate song;

[0039] Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs, respectively, to obtain the similarity between each event feature and all candidate songs;

[0040] The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

[0041] Furthermore, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method further includes:

[0042] Extracting all video frames of the target video to obtain multiple first video frames;

[0043] Performing median filtering on each of the first video frames to obtain multiple second video frames;

[0044] comparing each second video frame with an adjacent previous second video frame according to a video node corresponding to each second video frame, and identifying multiple pairs of motion vectors;

[0045] fusing each second video frame with all previous second video frames according to the multiple pairs of motion vectors and a weighted averaging algorithm to obtain an image sequence, wherein each weight of the weighted averaging algorithm is inversely proportional to the temporal distance between each second video frame and the first second video frame;

[0046] The fused image sequence is input into a pre-trained convolutional neural network model to obtain a denoised target video. The convolutional neural network model is used to restore the noisy image to a clean image.

[0047] Furthermore, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method further includes:

[0048] Filtering out a target event feature representing the main theme of the hot event from the multiple event features;

[0049] Determine the video segment corresponding to the target event feature to obtain the target video segment;

[0050] Convert each video frame of the target video clip into a grayscale image;

[0051] Convolving each of the grayscale images using a Laplacian operator to detect rapidly changing regions in each of the grayscale images;

[0052] Calculating the variance of the rapidly changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image.

[0053] The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

[0054] Preferably, converting each video frame of the target video clip into a grayscale image comprises:

[0055] Input each video frame of the target video clip into a pre-built OpenCV library;

[0056] The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

[0057] Furthermore, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method further includes:

[0058] Querying matching transition effects according to each event feature;

[0059] Add the corresponding transition effect between the original video and the inserted video clip.

[0060] The present invention also provides a video content automatic generation system, comprising:

[0061] An acquisition module is used to acquire hot events and original videos corresponding to the hot events;

[0062] An extraction module, configured to extract a plurality of event features describing the hot event;

[0063] A first query module is used to query a video library for candidate videos whose video content matches each of the event features, to obtain a plurality of target candidate videos;

[0064] A second query module is used to query the song library for candidate songs whose lyrics match each of the event features, to obtain multiple target candidate songs;

[0065] A generating module, configured to combine the target candidate video corresponding to each event feature with the corresponding target candidate song, thereby generating a video clip corresponding to each event feature;

[0066] The insertion module is used to determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0067] Compared with the prior art, the technical solution of the present invention has at least the following advantages:

[0068] The video content automatic generation method and system provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of hot events and matching them with the content in the video library and song library, thereby improving the attractiveness of the video and the audience's viewing interest; by combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between video images and music and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flowchart of an embodiment of the method for automatically generating video content of the present invention;

[0070] Figure 2 This is a flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0071] Figure 3 This is a flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0072] Figure 4 This is a flowchart of another embodiment of the method for automatically generating video content of the present invention;

[0073] Figure 5 This is a structural block diagram of an embodiment of the video content automatic generation system of the present invention. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0075] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as S11, S12, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0076] It will be understood by those skilled in the art that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the description of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0077] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0078] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0079] Please refer to Figure 1 The present invention provides a method for automatically generating video content, comprising the following steps:

[0080] S11, obtaining a hot event and the original video corresponding to the hot event;

[0081] S12, extracting multiple event features describing the hot event;

[0082] S13, searching the video library for candidate videos whose video content matches each of the event features, to obtain multiple target candidate videos;

[0083] S14, searching the song library for candidate songs whose lyrics match each of the event features, to obtain a plurality of target candidate songs;

[0084] S15, combining the target candidate video corresponding to each event feature with the corresponding target candidate song, to generate a video clip corresponding to each event feature;

[0085] S16: Determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0086] This embodiment can utilize technical means such as news aggregation services and social media monitoring tools to monitor and obtain hot events and related original videos in real time. Feature descriptors such as SIFT, SURF, or ORB are then used to extract key features from the original videos. These key features are used as event features to describe the hot events. These event features can describe the core content of the hot events and can be in the form of keywords, sentences, or paragraphs, such as keywords that describe the content attributes, dissemination characteristics, and social impact of the original videos.

[0087] Next, this embodiment uses a video and audio content matching algorithm to search for candidate videos that match each event feature from a pre-built video library. After deleting duplicate candidate videos, multiple target candidate videos are ultimately obtained. The video library pre-stores candidate videos that have been assigned tags, which characterize the video features of the corresponding candidate videos. For example, the tag of each candidate video in the video library is obtained, and each event feature is matched against the tag of each candidate video to determine a candidate video that matches each event feature.

[0088] Similarly, candidate songs matching each event feature are searched from a pre-built song library, and duplicate candidate songs are removed to ultimately obtain multiple target candidate songs. The song library also pre-stores candidate songs that have been assigned tags, which characterize the lyrical features of the corresponding candidate songs.

[0089] In addition, this embodiment combines the target candidate video corresponding to each event feature with the target candidate song corresponding to each event feature to generate a video clip related to the event feature, so as to enhance the emotional expression and visual impact of the video through matching music and video clips. Then, the system determines the corresponding video node of each event feature in the original video, inserts the generated video clip into the corresponding position of the original video, and forms the final target video, so that the generated video is highly relevant to the hot event, can attract the audience's attention, and increase viewing rate and participation; at the same time, videos closely related to hot events are also more likely to be disseminated on social media, optimizing the distribution effect of the content. In addition, the automated processing flow also reduces the workload of manual editing and shortens the time of video production.

[0090] For example, suppose there is a hot event called "music festival scene" and the original video is a live performance. The system identifies event features such as "music", "crowd", "stage lights" and other event features. Find live performance videos that match "music" and "stage lights" from the video library. Find popular songs that match "music festival scene" from the song library, combine the matched video clips with the popular songs to generate new video clips. In the original video, find the video nodes that match each event feature (such as the video node where the music ends, the part where the crowd quiets down, or the video node where the stage lights switch), and insert the newly generated video clips into the corresponding video nodes of the original video. In the end, a target video is obtained that includes both the original live performance and the new live video and music elements. This video is more vivid and rich, can better attract the audience's attention, and gain more dissemination on social media.

[0091] The video content automatic generation method provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of the hot events and matching them with the contents in the video library and the song library, thereby improving the attractiveness of the video and the audience's viewing interest; by combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between the video image and the music, and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce the production cost.

[0092] In one embodiment, please refer to Figure 2 The step of obtaining hot events may specifically include:

[0093] S111. Obtain user behavior data for each event within a preset time period, wherein the user behavior data includes clicks, shares, comments, and likes;

[0094] S112. Setting a weight corresponding to each type of user behavior data;

[0095] S113: performing weighted summation on the user behavior data of each event and the corresponding weight to obtain a popularity value of each event;

[0096] S114. Sort all events in descending order of the popularity values, and select the events with the top N popularity values ​​as hot events, where N is a positive integer.

[0097] Within a preset time period, such as 7 days, user behavior data for each event is collected from social media, news websites, forums and other platforms through data analysis tools or APIs. This user behavior data may include the number of clicks, shares, comments and likes of the original video, so as to fully reflect the user's attention and participation in different events, and provide basic data for subsequent heat analysis.

[0098] Furthermore, this embodiment can set weights for various types of user behavior data based on their contribution to the event's popularity. For example, clicks and shares may better reflect the reach of an event than likes and comments, and therefore can be given higher weights. This weighting can more accurately reflect the actual impact of different user behaviors on event popularity, making the calculated popularity value more scientific and reasonable.

[0099] Then, using the weighted summation method, the user behavior data corresponding to each event is multiplied by its corresponding weight, and the sum is calculated to obtain the popularity value of each event, so as to comprehensively consider the importance of different user behaviors and provide a quantitative heat index to help distinguish the popularity of events.

[0100] Finally, based on the calculated heat values, all events are sorted in descending order according to the corresponding heat values, and the events with the top N heat values ​​are selected as hot events. By sorting and screening, it can help quickly identify the most concerned social events.

[0101] For example, if you want to identify the hot news events of the week, your system can collect the following user behavior data from news websites and social media platforms:

[0102] Event A: 1000 clicks, 200 shares, 50 comments, and 300 likes;

[0103] Event B: 800 clicks, 150 shares, 75 comments, and 400 likes.

[0104] The system's weight for each type of user behavior data can be set as follows:

[0105] Clicks: 0.4;

[0106] Share amount: 0.3;

[0107] Comment volume: 0.2;

[0108] Likes: 0.1.

[0109] Then, the popularity value of each event is calculated based on the specific values ​​and corresponding weights of each type of user behavior data:

[0110] The heat value of event A is: 1000*0.4+200*0.3+50*0.2+300*0.1=500;

[0111] The heat value of event B is: 800*0.4+150*0.3+75*0.2+400*0.1=460.

[0112] Finally, all events are sorted in descending order of popularity. At this point, it can be found that the popularity value of event A is higher than that of event B. If N=1 is set, then event A is the hot event of this week. By quantifying the popularity of events, hot events can be accurately screened out.

[0113] In another embodiment, please refer to Figure 3 The step of obtaining hot events may specifically include:

[0114] S211, obtaining multiple events within a preset time period;

[0115] S212: Mark each of the events as a network node, and calculate the number of the network nodes to obtain the total number of network nodes;

[0116] S213. For each of the network nodes in the network, calculate the shortest path length from each of the network nodes to other network nodes in the network according to the Dijkstra algorithm;

[0117] S214: Summing the shortest path lengths from each network node to other network nodes in the network, to obtain a target shortest path length for each network node;

[0118] S215. Calculate the logarithm of the target shortest path length of each network node with the total number of network nodes as the base, to obtain a closeness centrality value of each network node;

[0119] S216 , screening out the events corresponding to the network nodes with the largest proximity centrality values ​​as hot events.

[0120] This embodiment, based on graph theory and network analysis methods, is used to identify hot events from multiple events. Specifically, data collection tools, such as news aggregation platforms and social media monitoring tools, can be used to collect all events that occurred within a specific time period. After screening, multiple events within a preset time period are obtained to ensure that the analyzed data is up-to-date and relevant, helping to capture timely hot events.

[0121] Next, in a mesh topology, each event is labeled as a network node. Each network node may be connected to two or more network lines. The node's position is determined by the lines it connects to and the other nodes in the network. The total number of network nodes is also counted. In graph theory, each event is treated as a node, and a network model is constructed. By converting events into nodes, graph theory methods can be used to analyze the relationships and influence between events.

[0122] For each node in the network, the Dijkstra algorithm is used to calculate the shortest path length from that node to all other nodes. The Dijkstra algorithm efficiently solves the shortest path problem in weighted graphs, providing the necessary path data for subsequent closeness centrality calculations. The Dijkstra algorithm is a graph search algorithm used to find the shortest path from a single source node to all other nodes in a weighted graph. The Dijkstra algorithm solves the single-source shortest path problem, which involves finding the shortest path from a given source node to all other nodes in the graph.

[0123] For each network node, the shortest path lengths from it to all other network nodes are summed to obtain the target shortest path length for that node. This target shortest path length can reflect the centrality of a node in the network, that is, the degree of proximity of the node to all other nodes.

[0124] Finally, using the total number of network nodes as the base, the logarithm of the target shortest path length is calculated for each network node to obtain the closeness centrality value of each network node. Based on the calculated closeness centrality value, the network node with the largest closeness centrality value is selected, and the corresponding event is designated as the hotspot event. This closeness centrality value quantifies the positional importance of the network node in the network. A larger value indicates that the node is closer to the network center, which can objectively identify the most influential hotspot events.

[0125] For example, suppose the system is analyzing news events from a week to determine which are hot topics. After filtering all news events, four news events from the week are collected. These four news events are marked as four network nodes (A, B, C, and D) in the mesh topology. If the Dijkstra algorithm is used to calculate the shortest path length from each network node to all other network nodes, the following is obtained:

[0126] The shortest path from A to B has a length of 1;

[0127] The shortest path from A to C has a length of 2;

[0128] The shortest path length from A to D is 3;

[0129] The shortest path length from B to C is 1;

[0130] The shortest path length from B to D is 2;

[0131] The shortest path from C to D has a length of 1.

[0132] From the above, we can see that for network node A, its target shortest path length is 1+2+3=6. With base 4 as the logarithm of the target shortest path length of network node A, log46, is calculated and used as the closeness centrality value of network node A. Similarly, the closeness centrality values ​​of the other three network nodes are calculated. The network node with the largest closeness centrality value is selected, and the corresponding event is identified as a hot event, thereby accurately identifying the most influential hot event from a large number of events.

[0133] In one embodiment, please refer to Figure 4 The step of searching for candidate videos whose video content matches each of the event features from the video library to obtain multiple target candidate videos may specifically include:

[0134] S131, obtaining the video content of each candidate video from the video library;

[0135] S132, extracting video features of the video content of each candidate video respectively, where the video features are used to represent semantic information of the video content of the corresponding candidate video;

[0136] S133, performing vectorization processing on each of the event features to obtain an event vector corresponding to each of the event features;

[0137] S134, performing vectorization processing on the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video;

[0138] S135, respectively calculating the cosine distance between the event vector corresponding to each event feature and the video vectors corresponding to all candidate videos, to obtain the similarity between each event feature and all candidate videos;

[0139] S136 , respectively screening out candidate videos with the greatest similarity corresponding to each of the event features, to obtain a plurality of target candidate videos.

[0140] In this embodiment, each candidate video is obtained from a pre-built video library, and the video content of each candidate video is extracted. A video analysis method (such as a 3D convolutional neural network (CNN)) is used to extract video features of the video content of each candidate video. These video features can represent the semantic information of the video content, such as the video type (entertainment, sports, education, etc.) described in the form of keywords, sentences, or text paragraphs, the scene type in the video (such as indoor, outdoor, specific location, etc.), or the object in the video (such as people, vehicles, animals, etc.). This video feature extraction can help understand the video content of the candidate videos and provide key information for subsequent video similarity calculations.

[0141] Next, we vectorize each event feature using one-hot encoding or a Word2Vec model to convert text, audio, or visual features into a numerical vector, known as an event vector. We also vectorize the extracted video features of each candidate video to obtain a video vector. This vectorization allows the event and video features to be used in mathematical calculations and comparisons, facilitating subsequent similarity calculations.

[0142] One-hot encoding (also known as single-bit encoding) is a method for converting categorical variables into a form that machine learning algorithms can better process. In one-hot encoding, each category value is represented as a binary vector with all zeros except for one position that is 1, representing the category.

[0143] The Word2Vec model is used to generate word embeddings. It learns the contextual relationships between words to generate word vectors that capture both semantic and syntactic information. The output of the Word2Vec model is a set of vectors, one for each word, with a fixed dimension. These vectors can be used in various natural language processing (NLP) tasks.

[0144] This embodiment then uses the cosine distance formula to calculate the cosine distance between each event vector and all candidate video vectors. This cosine distance is used to characterize the similarity between each event feature and each corresponding candidate video. This cosine distance measures the angle between two vectors; the smaller the angle, the higher the similarity. Finally, based on the calculated similarity, the candidate video with the greatest similarity to each event feature is selected. These selected candidate videos are the most closely matched to the event features, improving the accuracy and relevance of video retrieval.

[0145] For example, suppose you're making a short video about a "city marathon" and need to find the most relevant video clips from a video library. First, you retrieve all videos tagged "marathon" from the video library. Use a deep learning model to extract visual features from these videos, such as scenes and character movements. Then, you vectorize the event features of the "city marathon" (such as the city landscape and the running crowd). At the same time, you vectorize the video features, calculate the cosine distance between the "city marathon" event vector and each candidate video vector, and select the video with the smallest cosine distance, which is the most similar to the "city marathon" event features. This allows you to quickly filter out the most relevant video clips from a large number of videos, providing powerful visual material for short video production.

[0146] In one embodiment, the step of querying the song library for candidate songs whose lyrics match each of the event features to obtain a plurality of target candidate songs may specifically include:

[0147] Obtain the lyrics of each candidate song from the song library;

[0148] Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song;

[0149] Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features;

[0150] Vectorizing the lyrics features of the lyrics content of each candidate song to obtain a lyrics vector corresponding to each candidate song;

[0151] Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs, respectively, to obtain the similarity between each event feature and all candidate songs;

[0152] The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

[0153] This example extracts each candidate song from a pre-built song library, extracts the lyrics for each candidate song, and uses natural language processing (NLP) to extract semantic features of the lyrics, such as theme, sentiment, and keywords. This lyric feature extraction helps understand the deeper meaning of the lyrics and provides key information for subsequent similarity calculations.

[0154] Next, we vectorize each event feature using one-hot encoding or a Word2Vec model, converting text, audio, or visual features into numerical vectors, known as event vectors. We also vectorize the lyrics features of each candidate song to produce a lyric vector. This vectorization allows the event features to be used for mathematical calculations and comparisons, facilitating subsequent similarity calculations.

[0155] One-hot encoding (also known as single-bit encoding) is a method for converting categorical variables into a form that machine learning algorithms can better process. In one-hot encoding, each category value is represented as a binary vector with all zeros except for one position that is 1, representing the category.

[0156] The Word2Vec model is used to generate word embeddings. It learns the contextual relationships between words to generate word vectors that capture both semantic and syntactic information. The output of the Word2Vec model is a set of vectors, one for each word, with a fixed dimension. These vectors can be used in various natural language processing (NLP) tasks.

[0157] This embodiment then uses the cosine distance formula to calculate the cosine distance between each event vector and the lyric vectors of all candidate songs. This cosine distance is used to characterize the similarity between each event feature and each corresponding candidate song. This cosine distance measures the angle between two vectors; the smaller the angle, the higher the similarity. Finally, based on the calculated similarity, the candidate songs with the greatest similarity to each event feature are selected. These selected candidate songs best match the event features, improving the accuracy and relevance of song retrieval.

[0158] For example, suppose you're producing a special program about "graduation season" and need to find the most relevant lyrics from a song library. First, retrieve the lyrics of all candidate songs related to themes such as "graduation," "youth," and "farewell" from the song library. Then use NLP technology to extract features such as themes, emotions, and keywords from these lyrics. Vectorize the event features of "graduation season" (such as graduation, youth, and farewell), and also vectorize the lyrics features. Calculate the cosine distance between the "graduation season" event vector and the lyrics vector of each candidate song. Select the songs with the smallest cosine distance, whose lyrics are most similar to the "graduation season" event features. This allows you to quickly filter out the lyrics most relevant to the "graduation season" theme from a large number of songs, providing powerful music material for short video production.

[0159] In yet another embodiment, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method may further include:

[0160] Extracting all video frames of the target video to obtain multiple first video frames;

[0161] Performing median filtering on each of the first video frames to obtain multiple second video frames;

[0162] comparing each second video frame with an adjacent previous second video frame according to a video node corresponding to each second video frame, and identifying multiple pairs of motion vectors;

[0163] fusing each second video frame with all previous second video frames according to the multiple pairs of motion vectors and a weighted averaging algorithm to obtain an image sequence, wherein each weight of the weighted averaging algorithm is inversely proportional to the temporal distance between each second video frame and the first second video frame;

[0164] The fused image sequence is input into a pre-trained convolutional neural network model to obtain a denoised target video. The convolutional neural network model is used to restore the noisy image to a clean image.

[0165] This embodiment describes a video processing flow that aims to improve the quality of a target video through median filtering, motion vector recognition, image fusion, and deep learning noise reduction.

[0166] Specifically, images are first extracted frame by frame from the target video to form a series of independent video frames to obtain multiple first video frames, and each extracted first video frame is processed by applying a median filter to obtain multiple second video frames, so as to retain the edge information of the first video frame while removing noise, especially salt and pepper noise, from the first video frame, thereby improving the image quality of the first video frame.

[0167] Next, an optical flow algorithm compares two adjacent median-filtered frames of the second video to identify the motion vector between them—the change in pixel position between the two frames. This motion vector helps understand the motion information in the video and provides a basis for subsequent image fusion.

[0168] Optical flow is an algorithm used in computer vision to estimate the motion patterns of pixels in an image sequence. It is based on the principle of constant brightness, which states that the brightness of pixels remains constant between consecutive image frames, and the speed and direction of pixel movement are consistent. Using this information, the optical flow algorithm constructs a vector field, where each vector represents the speed and direction of movement of an object point in the image, known as a motion vector.

[0169] In one embodiment, each second video frame is compared with an adjacent previous second video frame to identify multiple pairs of motion vectors. Specifically, each second video frame can be aligned with the adjacent previous second video frame. Then, a corner detection algorithm, such as Harris corner detection or Shi-Tomasi corner detection, can be used to identify corners in each second video frame. Corners whose brightness change from the previous second video frame to the next second video frame is greater than a threshold are selected as feature points. The corner detection algorithm can identify regions with significant brightness changes, which are more likely to correspond to the same feature points of the same object in different frames in the image sequence.

[0170] For each detected feature point, an optical flow equation is constructed. This optical flow equation is based on the assumption of constant brightness, which states that the brightness of the same object remains constant in consecutive image frames. For each feature point, the optical flow vector is estimated based on the equation by minimizing the weighted sum of squared residuals, thereby identifying multiple pairs of motion vectors. This residual is the difference between the actual brightness value and the brightness value predicted based on the optical flow assumption.

[0171] Then, this embodiment can use a weighted average algorithm to fuse each processed second video frame with all previous second video frames according to the motion vector to obtain an image sequence. Specifically, the motion vector is used to align the current second video frame with the adjacent previous second video frame, and the alignment process involves adjusting the pixel position according to the motion vector. After alignment, the weighted average algorithm is used to fuse the current second video frame with the adjacent previous second video frame, and the fused video frame is then fused with all previous second video frames to finally obtain an image sequence. Each frame of the image sequence contains the information of all previous frames, thereby enhancing the continuity and integrity of the video content.

[0172] Each weight in the weighted averaging algorithm is inversely proportional to the corresponding temporal distance, which is the time interval between each second video frame and the first second video frame. It should be noted that when the second video frame is the first second video frame, the temporal distance is 0, and the corresponding weight is also 0. This step, through image fusion, can reduce flicker and discontinuity between frames, improving the stability and smoothness of the video.

[0173] Finally, the fused image sequence is fed into a pre-trained convolutional neural network model for denoising. This convolutional neural network model can learn complex image features and effectively restore a clean image from a noisy image.

[0174] This embodiment can significantly improve the quality of low-light videos, reduce noise, and make videos clearer and smoother through image processing methods such as median filtering, motion vector recognition, image fusion, and deep learning noise reduction.

[0175] In one embodiment, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the following steps may be further included:

[0176] Filtering out a target event feature representing the main theme of the hot event from the multiple event features;

[0177] Determine the video segment corresponding to the target event feature to obtain the target video segment;

[0178] Convert each video frame of the target video clip into a grayscale image;

[0179] Convolving each of the grayscale images using a Laplacian operator to detect rapidly changing regions in each of the grayscale images;

[0180] Calculating the variance of the rapidly changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image.

[0181] The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

[0182] This embodiment aims to filter out the features that best represent the hot event from multiple event features and select a video cover accordingly. Specifically, multiple event features are first analyzed to identify the core features that best represent the hot event. By filtering out the main features, the hot event can be more accurately located, providing key information for subsequent video processing and cover selection.

[0183] Then, according to the selected target event features, the corresponding video clips are extracted from the video as target video clips to ensure that the video clips are closely related to the hot events, thereby improving the pertinence and relevance of the video content.

[0184] Each frame in the video clip is then converted to a grayscale image to simplify the data and highlight the image's structural information. Grayscale image processing is simpler and less computationally intensive than color image processing, while preserving important image features such as edges and textures.

[0185] Apply the Laplacian operator to each grayscale image to detect rapidly changing regions, or edges, within the image. This Laplacian operator can highlight edge information in the image, facilitating subsequent image clarity assessment.

[0186] Furthermore, this embodiment calculates the variance of rapidly changing regions in the image processed by the Laplacian operator as a measure of image clarity. A larger variance indicates greater local contrast and clearer image quality, which helps select the most representative video frame for the cover. Finally, based on the calculated clarity, the video frame corresponding to the grayscale image with the highest resolution is selected as the video cover to attract audience attention and increase the video's click-through rate and view rate.

[0187] This embodiment can ensure that the video cover has high definition and accurately reflects the core content of the hot event, thereby improving the attractiveness and dissemination effect of the video.

[0188] In one embodiment, converting each video frame of the target video clip into a grayscale image includes:

[0189] Input each video frame of the target video clip into a pre-built OpenCV library;

[0190] The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

[0191] This example uses the OpenCV library to grayscale the video frames of a target video clip. Specifically, each video frame is read from the target video clip one by one and passed as input to the OpenCV library. The cvtColor() function in the OpenCV library is used to convert each video frame from a color image to a grayscale image. This grayscale conversion simplifies the video frames of the target video clip, reducing computational complexity while preserving the video frame's structural information.

[0192] Among them, OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. The OpenCV library contains hundreds of computer vision algorithms that can be used to process and analyze image and video data.

[0193] The cvtColor() function is a function in the OpenCV library that is used to convert the color space of an image. The cvtColor() function can change an image from one color space to another, such as from BGR (blue green red) to RGB (red green blue), or from a color image to a grayscale image.

[0194] In one embodiment, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the following steps may be further included:

[0195] Querying matching transition effects according to each event feature;

[0196] Add the corresponding transition effect between the original video and the inserted video clip.

[0197] This embodiment can match and add transition effects according to event characteristics during the video production process to enhance the visual effects and narrative fluency of the video.

[0198] Specifically, first, analyze the video content and event characteristics, and determine the appropriate transition effects to ensure that the transition effects match the emotions and rhythm of the video content, thereby improving the professionalism and viewing experience of the video.

[0199] Secondly, this embodiment can use video editing software or programming libraries (such as OpenCV) to add selected transition effects between the original video and the inserted video clip. The transition effects can smoothly connect the original video and the video clip, reduce visual jumps, and enhance the coherence and narrative fluency of the video.

[0200] For example, suppose you're editing a video about a city marathon and need to add transition effects between different segments. Based on the event characteristics of the city marathon, such as "start," "mid-race," and "finish," you can query the database to find suitable transition effects. For example, the "start" segment might match a "fast blur to sharp" transition effect, while the "finish" segment might match a "slow motion to normal speed" transition effect. Using OpenCV or a similar video processing library, you can write code to add this selected transition effect between the corresponding segments of the original video, reducing visual jumps and enhancing the video's coherence and narrative fluency.

[0201] Please refer to Figure 5 , an embodiment of the present invention further provides a video content automatic generation system, comprising:

[0202] An acquisition module 51 is configured to acquire a hot event and an original video corresponding to the hot event;

[0203] An extraction module 52 is used to extract multiple event features describing the hot event;

[0204] A first query module 53 is configured to query a video library for candidate videos whose video content matches each of the event features, thereby obtaining a plurality of target candidate videos;

[0205] The second query module 54 is used to query the song library for candidate songs whose lyrics match each of the event features, and obtain multiple target candidate songs;

[0206] A generating module 55 is configured to combine the target candidate videos corresponding to each event feature with the corresponding target candidate songs to generate a video clip corresponding to each event feature;

[0207] The insertion module 56 is used to determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video.

[0208] The video content automatic generation system provided by the present invention can ensure that the generated video content is highly relevant to the hot events by extracting multiple event features of hot events and matching them with the contents in the video library and song library, thereby improving the attractiveness of the video and the audience's viewing interest; by combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated, which can achieve similarity matching between video images and music, and improve the overall viewing experience and artistic effect of the video; by determining the video node corresponding to each event feature in the original video, the video clip corresponding to each event feature is inserted into the video node corresponding to the original video to generate the target video, which can greatly reduce the workload of manual editing, shorten the video production cycle, improve the creation efficiency of short videos, and reduce production costs.

[0209] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0210] In one embodiment, the present invention further provides a storage medium storing computer-readable instructions. When executed by one or more processors, the computer-readable instructions cause the one or more processors to perform the above-described method for automatically generating video content. The storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0211] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a storage medium, and when executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0212] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for automatically generating video content, characterized in that: include: Obtaining hot events and original videos corresponding to the hot events; Extracting multiple event features describing the hot event, wherein the event features are used to describe the core content of the hot event; Querying a video library for candidate videos whose video content matches each of the event features to obtain multiple target candidate videos; Searching for candidate songs whose lyrics match each of the event features from a song library to obtain a plurality of target candidate songs; After combining the target candidate video corresponding to each event feature with the corresponding target candidate song, a video clip corresponding to each event feature is generated; Determine the video node corresponding to each event feature in the original video, and insert the video clip corresponding to each event feature into the video node corresponding to the original video to generate a target video; The obtaining of hot events includes: Acquire multiple events within a preset time period, mark each event as a network node, and calculate the number of network nodes to obtain the total number of network nodes. For each network node in the network, calculate the shortest path length from each network node to other network nodes in the network according to the Dijkstra algorithm, sum the shortest path lengths from each network node to other network nodes in the network, and obtain the target shortest path length of each network node. Calculate the logarithm of the target shortest path length of each network node with the total number of network nodes as the base to obtain the closeness centrality value of each network node, and select the event corresponding to the network node with the largest closeness centrality value as the hot event; Wherein, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method further includes: Extract all video frames of the target video to obtain multiple first video frames, perform median filtering on each of the first video frames to obtain multiple second video frames, compare each second video frame with the adjacent previous second video frame according to the video node corresponding to each second video frame, identify multiple pairs of motion vectors, and fuse each second video frame with all previous second video frames according to the multiple pairs of motion vectors and a weighted averaging algorithm to obtain an image sequence, wherein each weight of the weighted averaging algorithm is inversely proportional to the time distance between each second video frame and the first second video frame, and input the fused image sequence into a pre-trained convolutional neural network model to obtain a denoised target video, and the convolutional neural network model is used to restore the noisy image to a clean image.

2. The method for automatically generating video content according to claim 1, wherein: Get hot events, including: Obtain user behavior data for each event within a preset time period, including clicks, shares, comments, and likes; Setting the weight corresponding to each type of user behavior data; After weighted summing the user behavior data of each event and the corresponding weight, the heat value of each event is obtained; All events are sorted in descending order of the heat values, and events with heat values ​​in the top N are selected as hot events, where N is a positive integer.

3. The method for automatically generating video content according to claim 1, wherein: Query the video library for candidate videos whose video content matches each of the event features, and obtain multiple target candidate videos, including: Obtain the video content of each candidate video from the video library; Extracting video features of the video content of each candidate video respectively, wherein the video features are used to represent semantic information of the video content of the corresponding candidate video; Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features; Performing vectorization processing on the video features of the video content of each candidate video to obtain a video vector corresponding to each candidate video; Calculating the cosine distance between the event vector corresponding to each event feature and the video vectors corresponding to all candidate videos, respectively, to obtain the similarity between each event feature and all candidate videos; The candidate videos with the greatest similarity corresponding to each of the event features are screened out respectively to obtain multiple target candidate videos.

4. The method for automatically generating video content according to claim 1, wherein: Search the song library for candidate songs whose lyrics match each of the event features, and obtain multiple target candidate songs, including: Obtain the lyrics of each candidate song from the song library; Extracting lyrics features of the lyrics content of each candidate song respectively, wherein the lyrics features are used to represent semantic information of the lyrics content of the corresponding candidate song; Performing vectorization processing on each of the event features respectively to obtain an event vector corresponding to each of the event features; Vectorizing the lyrics features of the lyrics content of each candidate song to obtain a lyrics vector corresponding to each candidate song; Calculating the cosine distance between the event vector corresponding to each event feature and the lyrics vectors corresponding to all candidate songs, respectively, to obtain the similarity between each event feature and all candidate songs; The candidate songs with the greatest similarity to each event feature are screened out respectively to obtain a plurality of target candidate songs.

5. The method for automatically generating video content according to claim 1, wherein: Inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video further includes: Filtering out a target event feature representing the main theme of the hot event from the multiple event features; Determine the video segment corresponding to the target event feature to obtain the target video segment; Convert each video frame of the target video clip into a grayscale image; Convolving each of the grayscale images using a Laplacian operator to detect rapidly changing regions in each of the grayscale images; Calculating the variance of the rapidly changing region in each of the grayscale images of the Laplacian operator to obtain the clarity of each of the grayscale images, wherein the variance is used as an indicator to measure the degree of response change of the grayscale image. The larger the variance, the clearer the corresponding grayscale image. The video frame corresponding to the grayscale image with the highest definition is selected as the video cover of the target video.

6. The method for automatically generating video content according to claim 5, wherein: Convert each video frame of the target video clip into a grayscale image, including: Input each video frame of the target video clip into a pre-built OpenCV library; The cvtColor() function in the OpenCV library is called to grayscale each of the video frames to obtain a grayscale image corresponding to each of the video frames.

7. The method for automatically generating video content according to claim 1, wherein: Inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video further includes: Querying matching transition effects according to each event feature; Add the corresponding transition effect between the original video and the inserted video clip.

8. A video content automatic generation system, characterized in that: include: An acquisition module is used to acquire hot events and original videos corresponding to the hot events; An extraction module, configured to extract a plurality of event features describing the hot event, wherein the event features are used to describe the core content of the hot event; A first query module is used to query a video library for candidate videos whose video content matches each of the event features, to obtain a plurality of target candidate videos; A second query module is used to query the song library for candidate songs whose lyrics match each of the event features, to obtain multiple target candidate songs; A generating module, configured to combine the target candidate video corresponding to each event feature with the corresponding target candidate song, thereby generating a video clip corresponding to each event feature; An insertion module is configured to determine a video node corresponding to each event feature in the original video, and insert a video clip corresponding to each event feature into a video node corresponding to the original video to generate a target video; The obtaining of hot events includes: Acquire multiple events within a preset time period, mark each event as a network node, and calculate the number of network nodes to obtain the total number of network nodes. For each network node in the network, calculate the shortest path length from each network node to other network nodes in the network according to the Dijkstra algorithm, sum the shortest path lengths from each network node to other network nodes in the network, and obtain the target shortest path length of each network node. Calculate the logarithm of the target shortest path length of each network node with the total number of network nodes as the base to obtain the closeness centrality value of each network node, and select the event corresponding to the network node with the largest closeness centrality value as the hot event; Wherein, after inserting the video clip corresponding to each event feature into the video node corresponding to the original video to generate the target video, the method further includes: Extract all video frames of the target video to obtain multiple first video frames, perform median filtering on each of the first video frames to obtain multiple second video frames, compare each second video frame with the adjacent previous second video frame according to the video node corresponding to each second video frame, identify multiple pairs of motion vectors, and fuse each second video frame with all previous second video frames according to the multiple pairs of motion vectors and a weighted averaging algorithm to obtain an image sequence, wherein each weight of the weighted averaging algorithm is inversely proportional to the time distance between each second video frame and the first second video frame, and input the fused image sequence into a pre-trained convolutional neural network model to obtain a denoised target video, and the convolutional neural network model is used to restore the noisy image to a clean image.

Images