Video automatic generation system based on Internet

Abstract

The present application discloses an Internet-based video automatic generation system, which uses artificial intelligence-based data analysis and processing technology to perform key frame sampling on the first video clip, and then performs picture content recognition and semantic distinguishable enhancement on each sampled first video clip image key frame, so as to obtain the type label of the first video clip according to the dynamic semantic representation of the picture content between each picture content semantically distinguishable enhancement feature. In this way, it helps to capture the core theme and important elements of the video, not just the most frequently appearing elements, and can capture the time evolution of the video content, so that the generated label not only reflects the static picture content, but also reflects the dynamic characteristics of the video.

Description

Technical Field

[0001] The present application relates to the field of intelligent generation, and more specifically, to an Internet-based automatic video generation system. Background Art

[0002] With the rapid development of Internet technology, video content has become an important form of information dissemination and entertainment consumption. Users' demand for personalized, high-quality video content is growing, which has prompted the development of video automatic generation technology. Patent CN118741247A proposes a video automatic generation method, including selecting local video clips and recording the total length and script information, generating a first label through frame-by-frame image recognition, matching and prioritizing templates according to labels, splicing high-priority templates with video clips, and adjusting the length to generate the final video.

[0003] In this patent, the video clip label is based on the image recognition results, counting the number of frames in which each element appears in the video clip, and selecting the element with the most frames as the label of the video clip. However, determining the label based only on the most frequently appearing element may not accurately capture the core content or theme of the video clip. For example, if an object frequently appears in the background of a video, but the actual main content is the action of the character, then the label may be biased towards the object in the background rather than the key content. In addition, since video content is usually dynamically changing, the number of occurrences of a single element may not be enough to represent the main content of the entire video clip, especially in videos with frequent scene transitions or rich action. This method may not accurately capture the core content of the video.

[0004] Therefore, an optimized video automatic generation system is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. An embodiment of the present application provides an Internet-based video automatic generation system, which performs key frame sampling on the first video clip by using artificial intelligence-based data analysis and processing technology, and then performs picture content recognition and semantically distinguishable enhancement on each sampled first video clip image key frame, so as to obtain the type label of the first video clip according to the dynamic semantic representation of the picture content between each picture content semantically distinguishable enhancement feature. In this way, it helps to capture the core theme and important elements of the video, not just the most frequently appearing elements, and can capture the temporal evolution of the video content, so that the generated label not only reflects the static picture content, but also reflects the dynamic characteristics of the video.

[0006] According to one aspect of the present application, there is provided an Internet-based video automatic generation system, which includes: a video clip extraction module, used to extract a first video clip from a local video database; a type label analysis module, used to analyze the first video clip to obtain a type label of the first video clip; a template determination module, used to extract template labels of multiple templates from a video material library, and based on the degree of match between the type label of the first video clip and the template labels of each template, prioritize the multiple template labels to obtain a plurality of prioritized sorting templates; a film generation module, used to extract the highest priority sorting template from the multiple sorting templates, and splice and cut the highest priority sorting template and the first video clip to obtain a film, characterized in that the type label analysis module The block includes: a video clip sampling unit, which is used to perform key frame sampling on the first video clip to obtain a time queue of first video clip image key frames; a picture content recognition unit, which is used to perform picture content recognition on each first video clip image key frame in the time queue of the first video clip image key frames to obtain a time queue of picture content key frame semantic coding features; a content enhancement unit, which is used to perform sequence condition distinguishable enhancement on the time queue of picture content key frame semantic coding features to obtain a time queue of picture content semantic distinguishable enhancement features; a type label generation unit, which is used to perform picture content dynamic encoding on the time queue of picture content semantic distinguishable enhancement features to obtain a picture content dynamic semantic coding representation, and obtain a type label of the first video clip based on the picture content dynamic semantic coding representation.

[0007] Compared with the prior art, the present application provides an Internet-based video automatic generation system, which uses artificial intelligence-based data analysis and processing technology to perform key frame sampling on the first video clip, and then performs picture content recognition and semantic distinguishable enhancement on each sampled first video clip image key frame, so as to obtain the type label of the first video clip according to the dynamic semantic representation of the picture content between the semantically distinguishable enhanced features of each picture content. In this way, it helps to capture the core theme and important elements of the video, not just the elements with the highest frequency, and can capture the time evolution of the video content, so that the generated label not only reflects the static picture content, but also reflects the dynamic characteristics of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 It is a block diagram of an Internet-based video automatic generation system according to an embodiment of the present application.

[0010] Figure 2 Schematic diagram of data flow of an Internet-based automatic video generation system according to an embodiment of the present application.

[0011] Figure 3 It is a block diagram of a type label analysis module in an Internet-based video automatic generation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0012] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0013] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0014] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0015] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0016] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0017] In the technical solution of the present application, an Internet-based automatic video generation system is proposed. Figure 1 It is a block diagram of an Internet-based video automatic generation system according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the Internet-based video automatic generation system according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, an Internet-based video automatic generation system 300 includes: a video clip extraction module 310, used to extract a first video clip from a local video database; a type label analysis module 320, used to analyze the first video clip to obtain a type label of the first video clip; a template determination module 330, used to extract template labels of multiple templates from a video material library, and based on the degree of match between the type label of the first video clip and the template labels of each template, prioritize the multiple template labels to obtain a plurality of prioritized sorting templates; a film generation module 340, used to extract the highest priority sorting template from the multiple sorting templates, and splice and cut the highest priority sorting template and the first video clip to obtain a film.

[0018] In particular, the video segment extraction module 310 is used to extract the first video segment from the local video database. The process of extracting the first video segment from the local video database is a key step in the Internet-based video automatic generation system, which ensures that subsequent processing (such as analysis, tag generation and final film synthesis) can be performed based on high-quality and relevant materials.

[0019] When a user needs to generate a new video, he or she will first browse the local video library through an intuitive and feature-rich user interface and manually select video clips according to his or her needs. This interface supports multiple filtering criteria, such as classification by folder, date range, video length, etc., so that users can quickly find the resources they need. For more advanced application scenarios, the system can also use machine learning algorithms for intelligent recommendations. This mechanism can automatically select the most suitable video clips for the current task based on the user's preference history, project requirements, and contextual information. Intelligent recommendations usually involve a deep understanding of the video content, including identifying features such as scenes, characters, and actions in the video to improve the relevance and accuracy of the recommendations.

[0020] Once the user has selected a video clip or the system has completed the automatic recommendation, the next step is to read and parse the metadata of the selected video in detail. The system will immediately read all available metadata of the selected video, which covers the basic properties and technical parameters of the video, such as file format (MP4, AVI), encoding standard (H.264, VP9), resolution (1080p, 4K), frame rate (FPS) and audio track information. In addition, if the video is accompanied by additional information such as the time and location of shooting, the system will also read it. If there is a script file, the system will parse it to obtain more guidance information about the structure and content of the video. The script file usually contains details such as the timing of the background music, subtitles, and the location of special effects insertion, which is crucial to ensure the quality and coherence of the generated video. The parsed script information will be integrated into the metadata of the video for use by subsequent processing modules.

[0021] In order to ensure that all video clips can be processed uniformly and meet the requirements of the target platform, the next step is to perform necessary preprocessing on the video. Considering that videos from different sources may have different formats and encoding methods, the system will convert them into a common format, such as an MP4 file encoded with H.264. This step ensures that all video clips can be processed under the same framework, simplifying the subsequent workflow. Remove unnecessary parts at the beginning and end of the video and edit the video according to specific editing rules. This step also includes operations such as adjusting the video size and cropping the screen ratio to better adapt to the needs of the template or target platform. At the same time, the system will perform a basic assessment of the video quality, detect and correct problems such as jitter and blur, and apply enhancement technology to improve the image quality when necessary. After completing the above preprocessing, the system will perform a quality check on each video clip to ensure that its clarity, stability and other aspects meet the expected standards. Only high-quality video clips that have passed rigorous inspection will enter the next stage of processing.

[0022] Calculating the total duration of the video clips and achieving script synchronization are key steps to ensure the smoothness and coherence of the final work. The system will add up the duration of all selected video clips to get the total duration of the entire project. This value is very important for subsequent splicing, editing, and ensuring that the final work meets the expected length. Accurate duration calculation helps to plan the overall structure of the video to avoid it being too long or too short. If there is a script file, the system will try to synchronize the video clips with the time nodes in the script. This means accurately matching key moments in the video with the actions, dialogues or music changes described in the script. In this way, the system ensures that the final output is not only visually coherent, but also achieves the best results in audio and other aspects.

[0023] In order to maintain the efficiency of the processing flow and the effective management of resources, the system also needs to carefully arrange the storage path of the video clips. In order to avoid affecting the original video file, the system will create temporary copies of the selected video clips in the workspace. These copies are used for the next processing steps and are automatically deleted after the task is completed. This not only protects the security of the source file, but also facilitates subsequent processing. The system records the exact storage location of each video clip in its local library, including the location of any temporary copies. This helps subsequent modules quickly access the required resources and maintain the efficiency of the processing flow. Accurate storage path management is particularly important for multi-module collaboration, which ensures smooth information transmission between modules.

[0024] Finally, in order for the video clip to smoothly enter the next stage - the type label analysis module, the system needs to prepare and send the corresponding data packet. All necessary information is packaged into a structured data object, which includes the video clip itself, metadata, preprocessing results, and any associated script information. This data packet design makes information transmission more orderly and efficient.

[0025] In particular, the type label analysis module 320 is used to analyze the first video segment to obtain the type label of the first video segment. Figure 3 As shown, the type label analysis module 320 includes: a video clip sampling unit 321, which is used to perform key frame sampling on the first video clip to obtain a time queue of the first video clip image key frames; a picture content recognition unit 322, which is used to perform picture content recognition on each first video clip image key frame in the time queue of the first video clip image key frames to obtain a time queue of picture content key frame semantic coding features; a content enhancement unit 323, which is used to perform sequence conditional distinguishable enhancement on the time queue of picture content key frame semantic coding features to obtain a time queue of picture content semantically distinguishable enhanced features; a type label generation unit 324, which is used to perform picture content dynamic encoding on the time queue of picture content semantically distinguishable enhanced features to obtain a picture content dynamic semantic coding representation, and obtain a type label for the first video clip based on the picture content dynamic semantic coding representation.

[0026] Specifically, the video clip sampling unit 321 is used to perform key frame sampling on the first video clip to obtain a time queue of key frames of the first video clip image. Considering that the first video clip contains key information or change points, these key information play an important role in the classification of the video. Based on this, in the technical solution of the present application, key frame sampling is performed on the first video clip to obtain a time queue of key frames of the first video clip image. In other words, key frame sampling is an effective method that allows the system to focus on those parts that are crucial to understanding the overall content of the video, while reducing unnecessary data processing work, helping to quickly grasp the main plot or theme of the video, making the entire video analysis process more efficient and targeted.

[0027] Specifically, the picture content recognition unit 322 is used to perform picture content recognition on each first video clip image key frame in the time queue of the first video clip image key frame to obtain a time queue of picture content key frame semantic coding features. Considering that each first video clip image key frame contains semantic information expressed by the picture content, such as identifying the main characters, activities, places and other elements in the video. The convolutional neural network model has a good ability to capture and mine key information in the image. Therefore, in the technical solution of the present application, each first video clip image key frame in the time queue of the first video clip image key frame is input into the picture content recognizer based on the convolutional neural network model to obtain a time queue of picture content key frame semantic coding feature vectors, so that high-level visual features can be effectively extracted from each key frame. These features usually contain abstract representations of complex information such as objects, scenes, textures, etc. in the image.

[0028] It is worth noting that the Convolutional Neural Network (CNN) is a deep learning model specifically designed to process data with a grid structure (such as images, videos, etc.). It performs well in the field of computer vision and is widely used in various tasks such as image classification, object detection, and semantic segmentation. Unlike traditional fully connected neural networks, CNN introduces convolutional layers to automatically extract local features in the data, and uses pooling layers to reduce the number of parameters and the amount of calculation, thereby improving the efficiency and generalization of the model. The structure of CNN consists of multiple parts. First, the input layer receives the original data, such as an image. For color images, the input layer is usually a three-dimensional tensor represented as width × height × number of channels (such as 3 channels for RGB images). Next is the convolution layer, each of which contains multiple convolution kernels that slide on the input data and perform convolution operations to generate feature maps. The size of the convolution kernel is usually a small square (such as 3×3 or 5×5), and each convolution kernel is responsible for capturing a specific type of local feature. After the convolution operation, an activation function (such as ReLU) is usually applied for nonlinear transformation to enhance the expressiveness of the model. In order to keep the size of the input and output feature maps consistent, additional zero-valued pixels (called "zero padding" or "padding") can be added around the input data. In addition, the pixel distance that the convolution kernel moves each time is called the stride. A larger stride can reduce the size of the output feature map. The convolution layer is usually followed by a pooling layer, which further compresses the data dimension by downsampling operations while retaining key features. Common pooling methods include max pooling and average pooling. Max pooling takes the maximum value in each small area as the representative value of the area, while average pooling takes the average value. The role of the pooling layer is to reduce the spatial size of the feature map and reduce the number of parameters while retaining the most important information. The feature map after multiple layers of convolution and pooling is flattened and sent to the fully connected layer, which flattens all the feature maps of the previous layer into a one-dimensional vector and then connects to several neurons. Each neuron is connected to all neurons in the previous layer, and the weights are updated through the back-propagation algorithm. The fully connected layer is used for the final classification or other decision-making tasks. The output layer is designed according to the specific task requirements and is usually used to output the prediction results. For classification tasks, the output layer will give the probability distribution of each category; for regression tasks, it directly outputs the value.

[0029] Specifically, the content enhancement unit 323 is used to perform sequence condition distinguishable enhancement on the time queue of the picture content key frame semantic coding features to obtain a time queue of picture content semantic distinguishable enhanced features. Considering that each picture content key frame semantic coding feature in the time queue of the picture content key frame semantic coding features has different degrees of deep-level important information structure. Therefore, in order to further enhance the expressiveness of each key frame semantic coding feature vector, so that the differences between the semantic coding features of each picture content key frame are more obvious, so as to better capture the subtle differences between different scenes or elements in the video clip, the present application introduces a sequence condition distinguishable enhancement mechanism to enhance the time queue of the picture content key frame semantic coding features to obtain a time queue of picture content semantic distinguishable enhanced features. In this way, the quality of a single key frame feature is improved, and the coherence and expressiveness of the entire sequence are enhanced, providing more accurate and rich data support for subsequent generation of type labels.

[0030] Specifically, the specific steps of performing serial condition distinguishable enhancement on the time queue of the picture content key frame semantic coding features include: first, performing field mapping on each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector to obtain a time queue of entry picture content key frame semantic coding feature vectors; that is, performing a field mapping operation on each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector to convert the picture content key frame semantic coding features into a form that is more suitable for the current model or task requirements, so that the features can better reflect the intrinsic relationship between samples or make it easier for the algorithm to capture key information.

[0031] More specifically, field mapping is performed on each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector using the following field mapping formula to obtain a time queue of the entry picture content key frame semantic coding feature vector; wherein the field mapping formula is: ;in, The first one in the time queue of the semantic encoding feature vector of the key frame of the picture content The semantic encoding feature vector of the key frame content of the picture, and are the first weight matrix and the second weight matrix respectively, yes The corresponding key frame semantic encoding feature vector of the entry screen content.

[0032] Next, the essential features of the time queue of the entry screen content key frame semantic coding feature vector are calculated to obtain the screen content key frame semantic field essential feature vector; in the embodiment of the present application, the specific calculation process includes: calculating the field depth factor of each entry screen content key frame semantic coding feature vector in the time queue of the entry screen content key frame semantic coding feature vector to obtain the time queue of the screen content key frame semantic field depth factor; in particular, the field depth factor can be understood as an indicator to measure the relative importance or complexity of the feature point in the area to which it belongs. After calculating the time queue of the screen content key frame semantic field depth factor, further based on the time queue of the screen content key frame semantic field depth factor, the screen content key frame semantic field essential feature vector of the time queue of the entry screen content key frame semantic coding feature vector is calculated. That is, a feature distribution that comprehensively reflects the core attributes of the time queue of the entry screen content key frame semantic coding feature vector is extracted using the field effect normalization mechanism to obtain the screen content key frame semantic field essential feature vector.

[0033] Among them, the specific process of calculating the field depth factor of each entry picture content key frame semantic coding feature vector in the time queue of the entry picture content key frame semantic coding feature vector includes: using a softmax function to process each entry picture content key frame semantic coding feature vector in the time queue of the entry picture content key frame semantic coding feature vector to obtain the time queue of the picture content key frame semantic field depth factor.

[0034] More specifically, the essential features of the time queue of the semantic coding feature vector of the entry screen content key frame are calculated by the following essential feature calculation formula to obtain the essential feature vector of the semantic field of the screen content key frame; wherein the essential feature calculation formula is: ;in, To calculate the square of the length of a vector, represents the logarithmic function value with base 2, For calculation The depth of field factor, for The corresponding keyframe semantic field depth factor of the picture content, yes function, for The corresponding standardized picture content keyframe semantic field depth factor, It is the essential feature vector of the semantic field of the key frame of the picture content.

[0035] Furthermore, the essential feature vector of the semantic field of the key frame of the picture content is used as a conditional feature vector, and each of the key frame of the picture content semantic coding feature vectors in the time queue of the key frame of the picture content semantic coding feature vector is distinguishably enhanced to obtain the time queue of the picture content semantic distinguishable enhanced coding vector as the time queue of the picture content semantic distinguishable enhanced feature. That is, the essential feature vector of the semantic field of the key frame of the picture content is used as additional conditional information, and the time queue of the picture content key frame semantic coding feature vector is distinguishably enhanced element by element, so as to specifically amplify the important features of certain aspects according to specific rules or weight adjustment schemes, and suppress other less important parts at the same time, so as to achieve the purpose of optimizing feature expression, so as to obtain the time queue of the picture content semantic distinguishable enhanced coding vector.

[0036] More specifically, the picture content key frame semantic field essential feature vector is used as the conditional feature vector, and each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector is distinguishably enhanced by the following distinguishable enhancement formula to obtain a time queue of picture content semantic distinguishable enhancement coding vectors as the time queue of picture content semantic distinguishable enhancement features; wherein the distinguishable enhancement formula is: ;in, is the semantic essential weight matrix of the key frame content of the picture, is the semantic weight matrix of the key frame of the local picture content, is the bias vector, yes function, It is the point multiplication by position. Yes The semantically enhanced picture content can be distinguished by an enhanced coding vector.

[0037] Specifically, the type label generating unit 324 is used to dynamically encode the time queue of the semantically distinguishable and enhanced features of the picture content to obtain the dynamic semantic coding representation of the picture content, and obtain the type label of the first video clip based on the dynamic semantic coding representation of the picture content. In an embodiment of the present application, first, the time queue of the picture content semantically distinguishable and enhanced coding vector is input into the picture content dynamic semantic coding vector based on the RNN model to obtain the picture content dynamic semantic coding vector; here, considering that each picture content semantically distinguishable and enhanced coding vector respectively expresses the picture content semantic information within a time point, therefore, in order to better understand the continuity and change trend between different key frames in the video clip, so as to more accurately represent the dynamic characteristics of the video content, in the technical solution of the present application, the time queue of the picture content semantically distinguishable and enhanced coding vector is input into the picture content dynamic semantic coding vector based on the RNN model to obtain the picture content dynamic semantic coding vector. That is to say, RNN can consider the information of the previous frame while processing the current frame, which helps to generate a coding vector containing more context information. This context-rich coding can help the system recognize more complex patterns, such as the development of actions, the transition of scenes, etc., so as to provide a more comprehensive and in-depth understanding of content.

[0038] It is worth noting that Recurrent Neural Network (RNN) is a type of neural network specifically designed to process sequence data. Unlike traditional feedforward neural networks, RNN has feedback connections that allow information to be passed between time steps. The basic idea of ​​RNN is to capture dependencies in sequences by introducing a "memory" mechanism. Specifically, it receives a new input at each time step and combines the information from all previous time steps for calculation. To achieve this, RNN has a hidden state at each time step, which depends not only on the current input, but also on the hidden state at the previous moment. Therefore, over time, RNN is able to accumulate and use past information to influence future outputs.

[0039] Subsequently, based on the dynamic semantic coding vector of the picture content, the type label of the first video clip is obtained. That is, in the technical solution of the present application, the dynamic semantic coding vector of the picture content is input into a classifier-based video recognizer to obtain the type label of the first video clip. That is, the dynamic semantic features of the picture content obtained by dynamic fusion of the time queue of the picture content semantically distinguishable enhanced coding vector are classified and processed to obtain the type label of the first video clip. In this way, it is helpful to capture the core theme and important elements of the video, not just the elements with the highest frequency of occurrence, and it is able to capture the temporal evolution of the video content, so that the generated label not only reflects the static picture content, but also reflects the dynamic characteristics of the video. Specifically, the specific process of inputting the dynamic semantic coding vector of the picture content into the classifier-based video recognizer includes: using multiple fully connected layers of the classifier to fully connect the dynamic semantic coding vector of the picture content to obtain a coding classification feature vector; passing the coding classification feature vector through the Softmax classification function of the classifier to obtain the type label of the first video clip.

[0040] Preferably, inputting the picture content dynamic semantic coding vector into a classifier-based video identifier to obtain the type label of the first video clip comprises: performing feature clustering on the feature set of the picture content dynamic semantic coding vector to obtain a picture content dynamic semantic coding class feature set and a picture content dynamic semantic coding class feature set, that is: ;in, represents each feature value in the feature set of the dynamic semantic coding class of the picture content, Represents each feature value in the feature set outside the dynamic semantic coding class of the picture content.

[0041] The ratio of the number of feature values ​​in the feature set of the picture content dynamic semantic coding class to the number of feature values ​​in the feature set of the picture content dynamic semantic coding vector is calculated, that is: ;in, represents the number of feature values ​​in the feature set of the dynamic semantic coding class of the picture content, The number of feature values ​​of the feature set representing the dynamic semantic coding vector of the picture content.

[0042] Calculate the sum of the absolute values ​​of all feature values ​​in the feature set of the dynamic semantic coding class of the picture content The sum of the absolute values ​​of all feature values ​​in the feature set of the dynamic semantic coding vector of the picture content The ratio of the powers is used to obtain the dynamic semantic coding modulation weight of the picture content, that is: ;in, Representing each feature value in the feature set of the dynamic semantic coding vector of the picture content, Represents the dynamic semantic coding modulation weight of the picture content.

[0043] Calculate the sum of squares of all feature values ​​in the feature set of the dynamic semantic coding class of the picture content The power of the sum of the squares of all feature values ​​in the feature set of the dynamic semantic coding vector of the picture content The ratio of the powers is used to obtain the dynamic semantic coding and reconciliation weight of the picture content, namely: ;in, Indicates the dynamic semantic coding and reconciliation weight of the picture content.

[0044] For each feature value in the feature set within the dynamic semantic coding class of the picture content, the product of the feature value and the dynamic semantic coding harmonic weight of the picture content is calculated, and then the product of the dynamic semantic coding modulation weight of the picture content is added to obtain the optimized feature value, that is: ;in, Represents the optimized feature value of each feature value in the feature set within the dynamic semantic coding class of the picture content.

[0045] For each feature value in the out-of-class feature set of the picture content dynamic semantic coding, the product of the feature value and the picture content dynamic semantic coding adjustment weight is calculated to obtain the optimized feature value, that is: ;in, Represents the optimized feature value of each feature value in the feature set outside the dynamic semantic coding class of the picture content.

[0046] The optimized picture content dynamic semantic coding vector based on the optimized feature values ​​of the picture content dynamic semantic coding in-class feature set and the picture content dynamic semantic coding out-class feature set is input into a classifier-based video identifier to obtain a type label of the first video clip.

[0047] Here, in the case where each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector respectively represents the image semantic features of each key frame, when performing feature distinguishability enhancement based on sequence condition description, the difference in sequence condition descriptive description caused by the image semantic differences between each key frame is taken into consideration, and it is expected to improve the semantically consistent aggregation expression effect of the picture content dynamic semantic coding vector obtained by feature distinguishability enhancement.

[0048] Therefore, while performing feature clustering on the dynamic semantic coding vector of the picture content, the interaction description of the key feature information of the dynamic semantic coding vector of the picture content in the clustering process is carried out, and the geometric equivariant topology of the feature is constructed by feature low-rank harmonic modulation based on the equivariance of the clustering features of the dynamic semantic coding vector of the picture content and the features as a whole, so as to obtain the translation and rotation symmetry of the schematic distribution of the clustering features of the dynamic semantic coding vector of the picture content relative to the features as a whole, thereby introducing geometric message passing into the feature expression of the dynamic semantic coding vector of the picture content and realizing the cluster mapping symmetry of the dynamic semantic coding vector of the picture content through manipulation of irreducible low-rank coefficients, thereby improving the consistency of the clustering-based feature representation of the dynamic semantic coding vector of the picture content, thereby improving the accuracy of the type label of the first video clip obtained by inputting the dynamic semantic coding vector of the picture content into a classifier-based video recognizer.

[0049] In particular, the template determination module 330 is used to extract template tags of multiple templates from the video material library, and prioritize the multiple template tags based on the matching degree between the type tag of the first video clip and the template tags of each template to obtain multiple prioritized sorting templates. In a specific example, after completing the selection, analysis and tag generation of the video clip, the next task is to find the template that best suits the video. To this end, the system first traverses the video material library to find templates with second tags that match the generated type tags. The "matching" here does not only refer to a simple keyword comparison, but is based on a more complex algorithm to evaluate the similarity between the two. Once several potential matching templates are found, the system will prioritize them according to preset rules. The sorting basis includes but is not limited to factors such as the historical frequency of use of the template, click-through rate, and user satisfaction. The recommendation algorithm will comprehensively consider these indicators and dynamically adjust the priority of each template so that the most likely to meet user needs and the most popular templates are ranked at the top. It is worth noting that this process is not static. It will continuously optimize itself with the arrival of new user data to ensure that the recommendation results are always close to current trends and individual preferences.

[0050] More specifically, the sorting criteria covers multiple aspects: Count the number of times each template has been selected in the past. A template with high frequency of use usually means that it has high versatility and popularity. Frequently used templates are often verified many times, proving that they can provide good results in most cases. Collect previous user evaluations of each template, and templates with high scores will be recommended first. User feedback directly reflects the actual performance and user experience of the template, and is an important indicator for measuring the quality of the template. Track the number of times each template has been clicked to view. A higher click-through rate indicates that users are more interested in the template. The click-through rate can be used as an intuitive reflection of the attractiveness of the template to help the system identify templates that are more popular with users. Based on the specific attributes of the video clips (such as duration, resolution, etc.), evaluate which templates are most suitable for current needs. Different video clips have different requirements, and the system needs to flexibly adjust the selection of templates to ensure the best adaptability. Combined with the current popular trends or hot topics, appropriately adjust the priority of certain templates to make them closer to the current market needs. The popularity trend of video content changes rapidly, and the system needs to have a certain degree of flexibility to respond to market changes in a timely manner. The recommendation algorithm will comprehensively consider all the above indicators and dynamically adjust the priority of each template, so that the most likely to meet user needs and the most popular templates are ranked at the top. It is worth noting that this process is not static. It will continuously optimize itself with the arrival of new user data to ensure that the recommendation results are always close to current trends and individual preferences.

[0051] In particular, the finished film generation module 340 is used to extract the sorting template with the highest priority from the multiple sorting templates, and splice and cut the sorting template with the highest priority and the first video clip to obtain a finished film. In a specific example, the highest priority template is spliced ​​with the original video clip to generate a complete finished film. This step also requires delicate operations. For example, when the duration of the template exceeds the original video, the system may automatically crop the video clip or use loop playback to adapt the template; on the contrary, if the template is shorter, additional content can be added in the appropriate position. Regardless of which method is adopted, the goal is to make the final work both beautiful and smooth. Once the finished film is generated, the system automatically saves the file and notifies the user that a new finished product is available. At the same time, users will be prompted to rate the templates used to collect valuable user experience feedback for future improvements in algorithms and services.

[0052] More specifically, after completing the tag matching and priority sorting of multiple templates in the video material library, the system will output a series of prioritized template lists. Next, the system will select the template with the highest priority as a candidate and further check its compatibility with the video clip. If the highest priority template fully meets the requirements, it will go directly to the next step; otherwise, the system will continue to select the next optimal template downward until the most suitable one is found. During the selection process, the system will consider factors such as technical specification adaptation, content consistency, and duration matching. Ensure that the technical parameters of the selected template (such as resolution, frame rate, audio format, etc.) match the video clip to avoid technical incompatibility issues; check whether the theme, style, and scene type of the template are consistent with the core content of the video clip to ensure that the final generated video is coherent and natural; evaluate whether the length of the template is suitable for the total duration of the video clip, and adjust the template or video clip if necessary to ensure the integrity and fluency of the finished video.

[0053] That is, once the most suitable template is selected, the system will perform necessary preprocessing on the template and video clips to prepare for subsequent splicing and editing. In order to ensure that the template and video clips can be processed under the same framework, the system will convert them into a common format, such as an MP4 file encoded in H.264. This step simplifies the subsequent workflow and improves processing efficiency. At the same time, unnecessary parts at the beginning and end of the video are removed, and the video is edited according to specific editing rules, including adjusting the video size, cropping the screen ratio, etc., to better adapt to the requirements of the template. In addition, the system will perform a basic assessment of the video quality, detect and correct problems such as jitter and blur, and apply enhancement technology to improve the image quality when necessary. If there is a script file, the system will try to synchronize the video clips with the time nodes in the script, accurately matching the key moments in the video with the actions, dialogues or music changes described in the script to ensure that the final output is not only visually coherent, but also achieves the best results in audio and other aspects.

[0054] After preprocessing, the system begins to splice and edit the highest priority template with the original video clip to generate a complete finished video. Based on the length of the video clip and the template, the system will accurately align the two on the timeline, including adjusting the playback speed of the video clip, repeating certain parts, or adding transition effects to ensure that the entire video is seamless in time. Considering that the template may contain specific opening animations, ending pictures, or other fixed elements, the system needs to ensure that these elements are coordinated with the visual style of the video clip, and make the visual transition between the video clip and the template smoother and more natural through color correction, filter application, etc. If the template contains background music or special effects sound, the system will mix these tracks with the sound track of the video clip, paying attention to keeping the volume balanced to avoid either side being too prominent and affecting the overall effect. In addition, the music rhythm can be dynamically adjusted according to the video content to make it more in line with the plot development. Some templates may come with various visual effects, such as transition effects, text titles, etc. The system will intelligently select and apply appropriate special effects according to the specific situation of the video clip to enhance the viewing and professionalism of the finished video. The splicing and editing stage is the key to producing high-quality videos. It involves not only technical operations, but also creative considerations to ensure that the final work can achieve the best effects in visual, auditory and emotional expression.

[0055] After the splicing and editing are completed, the system will conduct a final review of the generated finished video to ensure that it meets the expected standards. The review includes checking whether the video screen is smooth and there are no obvious jumps or abruptness; confirming that the sound is well synchronized with the picture, and the background music and commentary are properly coordinated; evaluating the overall look and feel of the finished video to ensure that it has a unified style, a clear theme, and clear expression. After a rigorous review, the system will save the final version in the specified format and notify the user that a new finished product is available. At the same time, users will be prompted to rate the templates used to collect valuable user experience feedback for future improvements to algorithms and services.

[0056] As described above, the video automatic generation system 300 based on the Internet according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an Internet-based video automatic generation algorithm. In a possible implementation, the video automatic generation system 300 based on the Internet according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the video automatic generation system 300 based on the Internet can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the video automatic generation system 300 based on the Internet can also be one of the many hardware modules of the wireless terminal.

[0057] Alternatively, in another example, the Internet-based video automatic generation system 300 and the wireless terminal may also be separate devices, and the Internet-based video automatic generation system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0058] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An Internet-based video automatic generation system, comprising: A video segment extraction module, used for extracting a first video segment from a local video database; A type label analysis module, configured to analyze the first video segment to obtain a type label of the first video segment; A template determination module, configured to extract template tags of a plurality of templates from a video material library, and prioritize the plurality of template tags based on the degree of matching between the type tag of the first video clip and the template tags of each template to obtain a plurality of prioritized sorting templates; A film generation module, used for extracting the sorting template with the highest priority from the multiple sorting templates, and splicing and cutting the sorting template with the highest priority and the first video clip to obtain a film, characterized in that the type label analysis module includes: a video clip sampling unit, used for sampling the key frames of the first video clip to obtain a time queue of the first video clip image key frames; a picture content recognition unit, used for performing picture content recognition on each first video clip image key frame in the time queue of the first video clip image key frames to obtain a time queue of picture content key frame semantic coding features; a content enhancement unit, used for performing sequence condition distinguishable enhancement on the time queue of the picture content key frame semantic coding features to obtain a time queue of picture content semantic distinguishable enhancement features; a type label generation unit, used for performing picture content dynamic encoding on the time queue of the picture content semantic distinguishable enhancement features to obtain a picture content dynamic semantic coding representation, and obtaining a type label of the first video clip based on the picture content dynamic semantic coding representation; Among them, the content enhancement unit includes: a field mapping subunit, which is used to perform field mapping on each picture content key frame semantic coding feature vector in the time queue of picture content key frame semantic coding feature vectors to obtain a time queue of entry picture content key frame semantic coding feature vectors; a picture content key frame semantic essence calculation subunit, which is used to calculate the essential characteristics of the time queue of the entry picture content key frame semantic coding feature vectors to obtain the picture content key frame semantic field essential feature vector; a distinguishable enhancement subunit, which is used to use the picture content key frame semantic field essential feature vector as a conditional feature vector, and respectively perform distinguishable enhancement on each picture content key frame semantic coding feature vector in the time queue of the picture content key frame semantic coding feature vector to obtain a time queue of picture content semantic distinguishable enhancement coding vectors as the time queue of picture content semantic distinguishable enhancement features.

2. The Internet-based video automatic generation system according to claim 1, characterized in that: The picture content recognition unit is used to: input each first video clip image key frame in the time queue of the first video clip image key frame into a picture content recognizer based on a deep neural network model to obtain a time queue of picture content key frame semantic coding feature vectors as the time queue of the picture content key frame semantic coding features.

3. The Internet-based video automatic generation system according to claim 2, characterized in that: The picture content identifier based on the deep neural network model is a picture content identifier based on the convolutional neural network model.

4. The Internet-based video automatic generation system according to claim 3, characterized in that: The picture content key frame semantic essence calculation subunit includes: a field depth factor calculation secondary subunit, which is used to calculate the field depth factor of each entry picture content key frame semantic coding feature vector in the time queue of the entry picture content key frame semantic coding feature vector to obtain the time queue of picture content key frame semantic field depth factors; a field essence feature calculation secondary subunit, which is used to calculate the picture content key frame semantic field essence feature vector of the time queue of the entry picture content key frame semantic coding feature vector based on the time queue of the picture content key frame semantic field depth factors.

5. The Internet-based video automatic generation system according to claim 4, characterized in that: The field depth factor calculation secondary subunit is used to: use a softmax function to process each entry picture content key frame semantic coding feature vector in the time queue of the entry picture content key frame semantic coding feature vector to obtain the time queue of the picture content key frame semantic field depth factor.

6. The Internet-based video automatic generation system according to claim 5, characterized in that: The type label generating unit comprises: a picture content dynamic encoding subunit, used for inputting the time queue of the picture content semantically distinguishable enhanced encoding vector into a picture content dynamic encoder based on an RNN model to obtain a picture content dynamic semantic encoding vector; and a first video clip type determining subunit, used for obtaining the type label of the first video clip based on the picture content dynamic semantic encoding vector.

7. The Internet-based video automatic generation system according to claim 6, characterized in that: The first video clip type determination unit is used to: input the picture content dynamic semantic coding vector into a classifier-based video identifier to obtain a type label of the first video clip.

Images