An AI-driven interactive short video generation construction method and system

Abstract

The application discloses an AI-driven interactive short video generation construction method and system, and relates to the field of short video generation, and comprises the following steps: obtaining historical creation data of multiple collaborators; extracting a style vector of each collaborator based on the historical creation data; performing weighted combination on the multiple style vectors to generate multiple style cross combination vectors; generating multiple short video candidate versions based on the style cross combination vectors; presenting the multiple short video candidate versions to a video audience and receiving feedback information of the video audience; determining an optimal style combination vector according to the feedback information; and generating a target short video based on the optimal style combination vector.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of short video generation, more particularly, to an AI-driven interactive short video generation construction method and system. BACKGROUND

[0002] With the rapid development of artificial intelligence, especially natural language processing, multi-modal modeling and generative adversarial networks, the digital content generation capability is constantly moving towards intelligence and personalization. In the emerging fields of short videos, music videos (MV), virtual concerts, etc., the generation method based on AIGC (AI Generated Content) has become an important means to improve the quality and efficiency of content creation. Related research and applications have widely introduced audio rhythm analysis, emotion recognition, cross-modal feature alignment and diffusion model generation technologies, aiming to realize the deep integration and dynamic matching of audio and visual information, so as to generate high-quality video content with consistent style and emotional expression.

[0003] For example, an intelligent MV generation method based on AIGC (CN119788886A) in the prior art processes the original music file and creation instructions (such as text keywords, reference images, style parameters) input by the user, extracts the rhythm and emotion features of the audio and aligns them with the semantic vector, further generates key frame sequences through a diffusion model and performs particle special effect enhancement, optical flow interpolation and long sequence prediction, and finally outputs video content with complete structure, clear style and audio-visual synchronization. This type of technology has achieved remarkable results in meeting the needs of single user input and personalized creation, providing ordinary users with fast and efficient video generation capabilities.

[0004] However, with the rapid development of short video platforms, the video content creation mode has gradually evolved from individual creation in the early stage to a mainstream trend of collaborative creation by multiple creators. In this collaborative process, it often involves multiple creators with distinct styles, such as directors, editors, editors, music composers in content teams, or big V bloggers on social platforms, brand content managers, etc. Due to natural differences in editing rhythm, color preference, emotional expression method, music selection, and language style between each creator, the style fusion problem and creation decision divergence brought by multiple participants have become one of the main bottlenecks in team cooperation.

[0005] Current AIGC video generation solutions (including the aforementioned patent) have deep processing and personalized generation capabilities for audio-visual information, but their design philosophy is still mainly aimed at single user or unified style input scenarios. In a multi-person collaboration mode, this technology fails to provide the following support capabilities:

[0006] Lack of structured modeling and fusion strategy for multi-style input vectors;

[0007] It is impossible to automatically generate and compare the multi-version style combinations;

[0008] It does not support automatic reconciliation and collaborative decision-making assistance based on style conflicts.

[0009] In practical applications, collaborators can only rely on offline discussions or manual trial-and-error methods to integrate styles, which not only affects collaboration efficiency, but also cannot guarantee the visual consistency and emotional tension of the final content.

[0010] Therefore, there is an urgent need for an intelligent generation method that supports multi-style extraction, multi-modal fusion, versioned generation, and feedback iteration optimization to address the style unification problem in a multi-creator collaboration context. This method should have the ability to extract style vectors, combine multiple styles, output multiple versions, and model subjective preference feedback to achieve an "experimental permutation and combination + high-quality fusion optimization" creative closed loop, thereby promoting AIGC technology from individual intelligent creation to a higher stage of multi-agent collaborative intelligent generation. SUMMARY

[0011] The technical problem to be solved by the present application is to provide an AI-driven interactive short video generation construction method and system to solve the problems mentioned in the background art.

[0012] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0013] An AI-driven interactive short video generation construction method, comprising the following steps:

[0014] Obtain historical creation data of multiple collaborators;

[0015] Based on the historical creation data, extract a style vector for each collaborator;

[0016] Weighted combination of the plurality of style vectors to generate a plurality of style cross-combination vectors;

[0017] Based on the style cross-combination vectors, generate a plurality of short video candidate versions;

[0018] Present the plurality of short video candidate versions to the video audience and receive feedback information from the video audience;

[0019] Determine the optimal style combination vector according to the feedback information;

[0020] Generate a target short video based on the optimal style combination vector.

[0021] In some embodiments, the style vector includes at least one of the following: video rhythm parameters, script language features, visual color tone features, audio emotion dimensions, subtitle style parameters, and shot frequency.

[0022] In some embodiments, the style cross-combination vector is defined by the following equation:

[0023]

[0024] wherein,

[0025] S k represents the kth combination style vector;

[0026] S i represents the style vector of the ith collaborator;

[0027] m is the number of collaborators;

[0028] α i ∈ [0, 1] is the weight factor of the corresponding ith collaborator, and satisfies

[0029] In some embodiments, the style cross-combination vector is generated by any one or more of the following strategies: equal-weight average combination, main style guiding combination, random disturbance combination, multi-point cross-combination, wherein different dimension parameters are respectively taken from the style vectors of different collaborators.

[0030] In some embodiments, the feedback information includes at least one of the following evaluations of the video audience on each candidate version: scoring, likes or dislikes, and text comments.

[0031] In some embodiments, the method further comprises a supervised learning process based on audience feedback to determine the style weight, specifically comprising:

[0032] Obtaining the style weight vector α = (α1, α2, …, α m ) corresponding to a plurality of short video candidate versions and the audience score results, and constructing a supervised learning training sample, the sample form being: the input is the style weight vector, and the output is the corresponding audience score value;

[0033] Training a regression model to fit the mapping relationship between the style weight vector and the audience score, to obtain a scoring function f(α);

[0034] In the test stage, an optimization algorithm is used to search for the optimal style weight

[0035] Outputting the optimal style combination vector S * :

[0036]

[0037] wherein, represents the ith collaborator style weight corresponding to the feedback optimal result.

[0038] In some embodiments, the video audience feedback process displays each candidate video in a side-by-side presentation and supports simultaneous scoring and labeling suggestions.

[0039] In some embodiments, the method further comprises saving the optimal style combination vector into a user preference database for subsequent fast call in the generation task.

[0040] The application also discloses an AI-driven interactive short video generation construction system, comprising:

[0041] A style extraction module is configured to extract style vectors from historical creation data of multiple collaborators;

[0042] A combination generation module is configured to generate multiple cross-combination vectors by weighted combination of the style vectors;

[0043] A video generation module is configured to generate corresponding short video candidate versions based on the cross-combination vectors;

[0044] A display and feedback module is configured to display the candidate versions to video audiences and receive feedback;

[0045] An optimization module is configured to generate an optimal style combination vector based on the feedback information.

[0046] In some embodiments, the system further comprises a storage module configured to store the generation results, user feedback information and the optimal style combination vector.

[0047] The application has the following advantages over the prior art: the method adopts an automatic style vector combination technique, supports multiple strategies (such as equal weight, main style orientation, etc.) to generate style cross-combination results, and quickly outputs multiple short video candidate versions with different styles. Compared with traditional manual coordination and repeated trial and error, this "experimental permutation and combination" method greatly shortens the creation cycle, enables the collaborators to quickly obtain diversified fusion schemes, thereby accelerating the decision-making process and improving the overall efficiency. Combined with the feedback information of the video audiences, it can more accurately determine which style is the best and avoid any biased judgment of a collaborator. In further embodiments, a regression model is used to more accurately obtain the optimal style weight and thus obtain the optimal style combination vector. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the overall flowchart of the application;

[0049] Figure 2 is the style extraction flowchart of the application;

[0050] Figure 3 is the style combination and video generation flowchart of the application;

[0051] Figure 4 is the feedback and optimization flowchart of the present application. DETAILED DESCRIPTION

[0052] The specific embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] As Figure 1 shown is the overall flowchart of the present application, the present application discloses an AI-driven interactive short video generation construction method and system, aiming to generate target short videos that meet the audience's preferences by fusing the creative styles of multiple collaborators and combining the feedback of video audiences.

[0054] The overall steps of the present application are as follows:

[0055] Obtain historical creation data of multiple collaborators;

[0056] Based on the historical creation data, extract the style vector of each collaborator;

[0057] Weighted combination of the multiple style vectors to generate multiple style cross combination vectors;

[0058] Based on the style cross combination vectors, generate multiple short video candidate versions;

[0059] Present the multiple short video candidate versions to the video audience and receive feedback information from the video audience;

[0060] Determine the optimal style combination vector according to the feedback information;

[0061] Generate target short videos based on the optimal style combination vector.

[0062] In a more specific embodiment, as Figure 2 shown, first, the historical creation data of multiple collaborators needs to be collected. These data usually come from the content they publish on video platforms, such as the pictures, audio, scripts, subtitles, etc. of short videos. The way to obtain these data can be through web crawlers to grab from public video platforms, or through the API interface provided by the platform to extract directly. For example, assuming there are three collaborators A, B, and C, who have published dozens of short videos on a certain video platform, the download links of these videos and their related metadata (such as title, script, publishing time, etc.) can be obtained through API, thus forming a data set containing multi-dimensional information. These data provide the basis for subsequent extraction of style features.

[0063] Based on the collected historical creation data, the invention quantitatively analyzes the creation style of each collaborator and generates a corresponding style vector. The style vector is a multi-dimensional numerical representation that covers multiple characteristics of the collaborator in video creation. Specifically, these characteristics include but are not limited to the following aspects:

[0064] Video rhythm parameter, reflecting the fast and slow rhythm of the video, usually measured by the number of shot changes per minute. For example, collaborator A's video changes 10 shots per minute, indicating a preference for fast editing.

[0065] Script language feature, describing the language style of the script, such as formal, humorous, lyrical, etc. Natural language processing techniques can be used to analyze the script's word choice and sentence structure, such as collaborator B's script using more witty short sentences, reflecting a humorous style.

[0066] Visual tone feature, representing the overall color tone of the video frame, such as warm tone (mainly red, yellow) or cool tone (mainly blue, gray). This can be obtained by analyzing the RGB value distribution of the video frame.

[0067] Audio emotion dimension, reflecting the emotional tendency of background music or voiceover, such as happy, sad, tense, etc. Audio processing models can extract features such as pitch and rhythm for classification.

[0068] Subtitle style parameter, including font (such as Songti, Arial), color, size, etc. For example, collaborator C prefers to use yellow bold subtitles.

[0069] Shot change frequency, similar to the video rhythm parameter, but focuses more on the regularity of the switch, such as whether it is evenly distributed.

[0070] In some embodiments, to convert these features into style vectors, machine learning models can be used. For example, use a convolutional neural network (CNN) to analyze video frames and extract visual tone features, subtitle style parameters, and shot change frequency; use a recurrent neural network (RNN) or Transformer model to process script text and extract language features; use audio processing tools (such as Librosa) to analyze the emotional dimension of the audio. The training process of these models is based on large-scale labeled data sets, such as video style classification data sets. After training, the model can map each collaborator's historical creation data into a fixed-length vector, for example, a vector containing 20 elements, where each element corresponds to a numerical representation of a style feature.

[0071] After extracting the style vector of each collaborator, the invention method generates multiple style cross-combination vectors through weighted combination. Specifically, the mathematical expression of the combination vector is:

[0072]

[0073] where S k represents the kth combined style vector, S i is the style vector of the ith collaborator, m is the number of collaborators, α i ∈[0,1] is the weight factor of the corresponding ith collaborator, and satisfies

[0074] By adjusting the weight α i , the fusion of different collaborator styles can be achieved. The weight factor α i is limited in the range of [0, 1], ensuring that the contribution of each collaborator is non-negative, and the constraint that the sum is equal to 1 ensures the standardization of the combination result, avoiding the numerical out-of-control of style features. For example, if α1=0.8, α2=0.1, α3=0.1, then the generated S k mainly reflects the style of collaborator 1, while slightly fusing the features of collaborators 2 and 3.

[0075] As shown in Figure 3 , in order to increase the diversity of style combination, the present application uses the following strategies to set the weight α i :

[0076] Equal-weighted average combination of all collaborators has equal weights, i.e. α i =1 / m. For example, when (m=3), (α1=α2=α3=1 / 3). The combination vector generated in this way is the average of all collaborator styles, suitable for exploring the effect of balanced fusion.

[0077] Main style-oriented combination of a certain collaborator's weight is significantly higher than that of other collaborators. For example, α1=0.8, α2=0.1, α3=0.1. In this way, the combination vector is closer to the style of the collaborator with the largest weight, suitable for highlighting the characteristics of a certain collaborator.

[0078] Random disturbance adds random disturbance to a certain base weight (such as equal weight) and then normalizes. For example, the initial value is α1=α2=α3=1 / 3, and a random number (such as 0.1, -0.05, -0.05) is added respectively, and then the sum is adjusted to 1. This way can increase the diversity of style and explore unexpected combination effects.

[0079] Multi-point intersection combines style parameters of different dimensions from different collaborator vectors. For example, the video rhythm parameter comes from collaborator 1, the script language feature comes from collaborator 2, and the visual tone feature comes from collaborator 3. This way is similar to the crossover operation in genetic algorithm, which can generate highly innovative style combinations.

[0080] Through these strategies, the system can generate multiple different S k , providing rich candidate styles for subsequent video generation.

[0081] Based on the generated style cross-combination vector S k , the invention further utilizes a generative model to generate corresponding short video candidate versions. Common generative models include Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE). Taking GAN as an example, its architecture includes a generator and a discriminator:

[0082] The generator is used to receive the style vector S k as input to generate video content. The generator can be a deep neural network that outputs a sequence of video frames, a script text, and an audio clip.

[0083] The discriminator is used to determine whether the generated content meets the distribution characteristics of real videos, guiding the training of the generator.

[0084] When training GAN, a short video dataset containing multiple styles can be used to ensure that the model can generate different S k to generate diverse videos. For example, input S1 = 1 / 3 S A + 1 / 3 S B + 1 / 3 S C , the generator may output a short video with moderate rhythm, balanced color, and humorous language. The generation process usually requires a pre-prepared material library (such as video clips, audio samples), and the model selects and combines content from the material library according to the guidance of S k .

[0085] After generating multiple short video candidate versions, the system presents them to the video audience and collects feedback information. To facilitate comparison, the display method uses side-by-side presentation, i.e., multiple videos are displayed on the screen at the same time. The audience can provide feedback in the following ways:

[0086] Scoring, scoring each video, for example, 1 to 10.

[0087] Like or dislike, simply indicating like or dislike.

[0088] Text comments, providing specific suggestions such as "rhythm too fast" or "color warmer".

[0089] Assuming there are 5 candidate videos V1 to V5, the audience's scores are 7, 8, 6, 5, and 9, respectively, and the system will record these feedbacks to optimize the style combination. The optimal style combination can be selected according to the highest score.

[0090] For example, Figure 4In another embodiment, a regression model can be employed to determine a more accurate optimal style combination vector S * The specific steps include:

[0091] First, a plurality of sets of style weight vectors of short video candidate versions and their corresponding audience feedback data are collected. Each candidate video corresponds to a determined style weight vector a = (a1, a2, …, a m This vector records the fusion proportion of multiple collaborator styles in the video, for example, a certain video may contain 30% of style A, 50% of style B and 20% of style C. At the same time, the system collects the feedback of the audience on these candidate videos, which adopts a numerical scoring form, such as scoring grades (such as 1 to 5 points), the number of likes or other indicators that can quantify audience preferences. These style weight vectors and corresponding score data together constitute the training samples required for supervised learning, in which the style weight vector a is the input and the audience satisfaction score is the output.

[0092] Using these training samples, the system trains a regression model, the goal of which is to learn a score function f(a), which can predict the audience's satisfaction with the style combination generated video according to the input style weight vector. By fitting the relationship between style weight and score, the regression model can capture the influence of different style proportions on audience preferences, for example, it may be found that a certain style combination is more popular. After training, the system enters the actual application stage, at which time multiple candidate videos are no longer needed to be generated to explore the audience's reaction, but the trained score function f(a) is directly used to search in the style weight space to find the optimal style weight combination This search process uses a continuous optimization method, such as the gradient descent algorithm, to iteratively adjust the value of a, gradually approaching the optimal solution, ensuring that the found can maximize audience satisfaction.

[0093] Calculate the optimal style combination vector According to the optimized weight, calculate This vector represents the most satisfying style combination of the audience.

[0094] Based on S * The system calls the generation model again to generate the final target short video. Since S * has fused the audience preferences, the generated video can better meet the needs. For example, if S * is biased towards the style of collaborator C, the target video may have high-frequency camera cuts and yellow subtitles.

[0095] This method significantly reduces the cost of traditional trial-and-error methods, as it no longer relies on repeatedly generating candidate videos and collecting feedback, but directly locks the best style combination through model prediction and optimization, thereby improving the efficiency and quality of style fusion. The entire process from data-driven model training to mathematical optimization of weight search embodies the strong application potential of supervised learning in short video content generation.

[0096] In another embodiment, after generating the target short video, the application will save the S * to the user preference database. This database can be a key-value pair storage system, where the key is the user ID and the value is the corresponding S * In subsequent tasks, the system can directly call the saved S * without re-optimization, thereby improving generation efficiency.

[0097] To implement the above method, the system of the application includes the following modules:

[0098] a style extraction module for extracting style vectors from historical creation data of multiple collaborators;

[0099] a combination generation module for weighted combination of the style vectors to generate multiple cross-combination vectors;

[0100] a video generation module for generating corresponding short video candidate versions based on the cross-combination vectors;

[0101] a display and feedback module for displaying candidate versions to video audiences and receiving feedback;

[0102] an optimization module for generating an optimal style combination vector based on feedback information.

[0103] a storage module for storing generation results, user feedback information, and optimal style combination vectors.

[0104] Through the method and system of the application, the styles of multiple collaborators are fused while meeting the individualized needs of the audience, demonstrating the strong potential of AI in short video creation.

[0105] The above description is only a preferred embodiment of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solution and inventive concept of the application within the scope of the disclosed technology, which should be covered within the protection scope of the application.

Claims

1. An AI-driven interactive short video generation construction method, characterized in that, The method comprises the following steps: obtaining historical creation data of a plurality of collaborators; extracting a style vector of each collaborator based on the historical creation data; performing weighted combination on a plurality of style vectors to generate a plurality of style cross-combination vectors; generating a plurality of short video candidate versions based on the style cross-combination vectors; presenting the plurality of short video candidate versions to a video audience and receiving feedback information of the video audience; determining an optimal style combination vector according to the feedback information; generating a target short video based on the optimal style combination vector; The style cross-combination vector is defined by the following formula: ； wherein, denotes the k-th combined style vector; style vector representing the ith collaborator; m is the number of collaborators; is the weight factor for the corresponding ith collaborator, and satisfies ; The method further comprises a supervised learning process based on audience feedback to determine style weights, specifically comprising: Obtaining a style weight vector corresponding to each of a plurality of short video candidate versions With the audience score result, a supervised learning training sample is constructed, which is in the form of: the input is the style weight vector, and the output is the corresponding audience score value; training a regression model to fit a mapping between the style weight vector and the audience score to obtain a scoring function ; In the testing phase, an optimization algorithm is employed to search for the optimal style weights that maximize the score function ;​ Output optimal style combination vector : ； wherein, represents the i-th collaborator style weight corresponding to the feedback optimal result.

2. The method of claim 1, wherein, The style vector comprises at least one of the following: video rhythm parameters, script language features, visual color tone features, audio emotion dimensions, subtitle style parameters, and shot switching frequency.

3. The method of claim 1, wherein, The style cross-combination vector is generated using any one or more of the following strategies: equal weight average combination, main style guiding combination, random disturbance combination, and multi-point cross-combination, wherein different dimension parameters are respectively taken from the style vectors of different collaborators.

4. The method of claim 1, wherein, The feedback information comprises at least one of the following evaluations of the video audience on each candidate version: scoring, likes or dislikes, and text comments.

5. The method of claim 1, wherein, The video audience feedback process displays each candidate video in a side-by-side presentation mode and supports simultaneous scoring and suggestion labeling.

6. The method of claim 1, wherein, The method further comprises saving the optimal style combination vector to a user preference database for quick calling in subsequent generation tasks.

7. An AI-driven interactive short video generation construction system, characterized in that, Comprise: a style extraction module for extracting style vectors from historical creation data of a plurality of collaborators; a combination generation module for performing weighted combination on the style vectors to generate a plurality of cross-combination vectors; a video generation module for generating corresponding short video candidate versions based on the cross-combination vectors; a display and feedback module for displaying candidate versions to a video audience and receiving feedback; an optimization module for generating an optimal style combination vector based on feedback information; The cross-combination vector is defined by the following formula: ； wherein, denotes the k-th combined style vector; represents the style vector of the ith collaborator; m is the number of collaborators; is the weight factor for the corresponding ith collaborator, and satisfies ; The supervised learning process based on audience feedback to determine style weights, specifically comprising: Obtaining a style weight vector corresponding to each of a plurality of short video candidate versions With the audience score result, a supervised learning training sample is constructed, which is in the form of: the input is the style weight vector, and the output is the corresponding audience score value; training a regression model to fit a mapping between the style weight vector and the audience score to obtain a scoring function ; In the testing phase, an optimization algorithm is employed to search for the optimal style weights that maximize the score function ;​ Output optimal style combination vector : ； wherein, represents the i-th collaborator style weight corresponding to the feedback optimal result.

8. The system of claim 7, wherein, The system further comprises a storage module for storing generation results, user feedback information, and optimal style combination vectors.

Images