Multi-agent-based automatic generation method, system and terminal for micro-drama

Abstract

The invention discloses a multi-agent-based automatic generation method, system and terminal for a micro-drama, and the method comprises the steps: obtaining a user creation theme, and carrying out the story line generation processing and the split processing to obtain a plurality of split scripts which comprise commentaries, visual information, shot information and atmosphere information; performing dubbing conversion processing and subtitle generation processing according to the commentary to obtain a target dubbing audio and a target video subtitle; performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing with the shot information to obtain a target video clip; performing background music generation processing according to the atmosphere information to obtain target background music; and performing time axis alignment processing and splicing processing on the target dubbing audio, the target video subtitles, the target video clips and the target background music to obtain a target synthesized episode. According to the invention, the production efficiency and the output quality of the micro-movie can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, terminal and computer-readable storage medium for automatically generating short plays based on multi-agents. Background Art

[0002] A micro-short drama refers to an online film or television work with a single episode length ranging from tens of seconds to about 15 minutes, with a relatively clear theme and main line, and a relatively continuous and complete storyline. The subject matter of micro-short dramas is very broad and can meet the needs of different audience groups.

[0003] However, the production process of traditional short dramas in the existing technology is too long. From script creation to film release, the entire process takes weeks or even months. It is impossible to respond quickly to market demands and hot events, and cannot meet user needs.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for automatically generating micro-short dramas based on multi-agents, aiming to solve the problem that the production process cycle of traditional micro-short dramas in the prior art is too long, resulting in the inability to quickly respond to market demands and hot events and the inability to meet user needs.

[0006] To achieve the above object, the present invention provides a method for automatically generating short plays based on multi-agents, and the method for automatically generating short plays based on multi-agents comprises the following steps:

[0007] Acquire a user-created theme, and perform storyline generation and storyboard processing according to the user-created theme to obtain a plurality of storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information, and atmosphere information;

[0008] Perform dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles;

[0009] Performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip;

[0010] Perform background music generation processing according to the atmosphere information to obtain target background music;

[0011] The target dubbing audio, the target video subtitles, the target video clips and the target background music are subjected to time axis alignment and splicing processing to obtain a target synthetic skit.

[0012] Optionally, the method for automatically generating short plays based on multi-agents, wherein the step of obtaining a user creation theme, and performing storyline generation and storyboard processing according to the user creation theme to obtain multiple storyboard scripts, specifically includes:

[0013] Acquire a user-created theme, and input the user-created theme into a story generation agent, and perform storyline generation processing on the user-created theme by the story generation agent to obtain a complete storyline;

[0014] The complete storyline is input into a storyboard generation agent, and the storyboard generation agent is used to storyboard the complete storyline to obtain a plurality of storyboard scenes and a storyboard script corresponding to each storyboard scene.

[0015] Optionally, the multi-agent-based automatic generation method for short short plays, wherein the step of obtaining a user creation theme, and performing storyline generation and storyboard processing according to the user creation theme to obtain a plurality of storyboard scripts, further comprises:

[0016] Acquire story data and script data, and pre-process the story data and the script data to obtain a training set, wherein the pre-processing includes denoising, format conversion, segmentation, and annotation;

[0017] A preset large language model is determined, and a cross-validation method is used to perform model training and model fine-tuning on the preset large language model according to the training set to obtain the story generation agent.

[0018] Optionally, the multi-agent-based automatic generation method for short short plays, wherein the dubbing conversion processing and subtitle generation processing are performed according to the commentary to obtain target dubbing audio and target video subtitles, specifically includes:

[0019] Input the commentary into the commentary dubbing generation agent and the subtitle generation module respectively;

[0020] The commentary dubbing generating agent performs text processing and language feature analysis on the commentary to obtain target language features, wherein the text processing includes text normalization, word segmentation and part-of-speech tagging, and the language feature analysis includes grammatical analysis, semantic analysis and prosody prediction;

[0021] Inputting the target language features into an acoustic model, extracting acoustic parameters of the target language features through the acoustic model to obtain target acoustic parameters, and inputting the target acoustic parameters into a vocoder to output target dubbing audio;

[0022] The subtitle generation module performs subtitle generation processing on the commentary to obtain target video subtitles, wherein the subtitle generation processing includes commentary reception, timeline allocation, subtitle line division, subtitle formatting and subtitle output.

[0023] Optionally, the multi-agent-based automatic generation method of short plays, wherein the step of performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip, specifically includes:

[0024] Inputting the visual information into a key frame generation agent, and performing key frame generation processing on the visual information by the key frame generation agent to obtain a target key frame image;

[0025] The target key frame image and the lens information are input into a video generation agent, and the video generation agent performs video generation processing according to the target key frame image and the lens information to obtain a target video clip.

[0026] Optionally, the method for automatically generating a short play based on a multi-agent, wherein the step of generating background music according to the atmosphere information to obtain target background music specifically includes:

[0027] Inputting the atmosphere information into a background music generation agent, and extracting key atmosphere information from the atmosphere information through the background music generation agent, wherein the key atmosphere information includes emotional tone, scene characteristics, and character status;

[0028] The atmosphere key information is subjected to music element conversion processing and music element matching processing to obtain target background music.

[0029] Optionally, the multi-agent-based automatic generation method of micro-short dramas, wherein the target dubbing audio, the target video subtitles, the target video clips and the target background music are subjected to timeline alignment and splicing processing to obtain the target synthetic short drama, specifically includes:

[0030] Inputting the target dubbing audio, the target video subtitles, the target video clips and the target background music into a skit synthesis module;

[0031] The target dubbing audio and the target background music are mixed with the target video clip by the skit synthesis module using audio processing technology, and audio editing and audio optimization are performed to obtain a first video clip;

[0032] Superimposing the target video subtitles onto the first video segment according to the timeline to obtain a second video segment;

[0033] The second video clips are spliced ​​using video editing technology to obtain a target synthetic short drama.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a multi-agent-based automatic generation system for short plays, wherein the multi-agent-based automatic generation system for short plays comprises:

[0035] A storyboard script generation module is used to obtain a user-created theme, and perform storyline generation and storyboard processing according to the user-created theme to obtain multiple storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information, and atmosphere information;

[0036] An audio subtitle generation module is used to perform dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles;

[0037] A video clip generation module, configured to perform key frame generation processing according to the visual information to obtain a target key frame image, and perform video generation processing according to the target key frame image and the lens information to obtain a target video clip;

[0038] A background music generation module, used to generate background music according to the atmosphere information to obtain target background music;

[0039] The synthetic skit generation module is used to perform timeline alignment and splicing processing on the target dubbing audio, the target video subtitles, the target video clips and the target background music to obtain a target synthetic skit.

[0040] In the present invention, a user creation theme is obtained, and a plot generation process and a storyboard process are performed according to the user creation theme to obtain multiple storyboard scripts, wherein the storyboard script includes commentary, visual information, lens information and atmosphere information; dubbing conversion processing and subtitle generation processing are performed according to the commentary to obtain target dubbing audio and target video subtitles; key frame generation processing is performed according to the visual information to obtain a target key frame image, and video generation processing is performed according to the target key frame image and the lens information to obtain a target video clip; background music generation processing is performed according to the atmosphere information to obtain target background music; the target dubbing audio, the target video subtitle, the target video clip and the target background music are time-aligned and spliced ​​to obtain a target synthetic short play. Through the collaborative work of multiple professional AI agents, the present invention can automatically complete the entire process from plot generation, storyboard design, audio-visual material synthesis to the final short play generation according to the creative theme input by the user, greatly improving the production efficiency and creative freedom of micro short plays, and realizing the full process automation of micro short play production. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flowchart of a preferred embodiment of the method for automatically generating short plays based on multiple agents of the present invention;

[0042] Figure 2 It is a schematic diagram of the overall implementation process of a preferred embodiment of the method for automatically generating short plays based on multiple agents of the present invention;

[0043] Figure 3 It is a schematic diagram of the implementation process of the story generation module of the preferred embodiment of the method for automatically generating short plays based on multi-agents of the present invention;

[0044] Figure 4 It is a schematic diagram of the implementation process of the storyboard generation module of a preferred embodiment of the multi-agent based micro-short drama automatic generation method of the present invention;

[0045] Figure 5 It is a schematic diagram of the implementation process of the narration dubbing generation module of the preferred embodiment of the multi-agent based micro-short drama automatic generation method of the present invention;

[0046] Figure 6 It is a schematic diagram of the implementation process of the subtitle generation module of a preferred embodiment of the multi-agent based micro-short drama automatic generation method of the present invention;

[0047] Figure 7 It is a schematic diagram of the implementation process of the key frame generation module of the preferred embodiment of the multi-agent based micro-short drama automatic generation method of the present invention;

[0048] Figure 8It is a schematic diagram of the implementation process of the text diffusion model of a preferred embodiment of the method for automatically generating short plays based on multi-agents of the present invention;

[0049] Fig. 9 It is a schematic diagram of the implementation process of the video clip generation module of the preferred embodiment of the multi-agent based micro-short drama automatic generation method of the present invention;

[0050] Fig.10 It is a schematic diagram of the implementation process of the background music module of a preferred embodiment of the method for automatically generating short plays based on multiple agents of the present invention;

[0051] Fig.11 It is a schematic diagram of the implementation process of the skit synthesis module of the preferred embodiment of the multi-agent based micro-skit automatic generation method of the present invention;

[0052] Fig.12 It is a structural diagram of a preferred embodiment of the multi-agent based micro-drama automatic generation system of the present invention;

[0053] Fig.13 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] A micro-short drama refers to an online film or television work with a single episode length ranging from tens of seconds to about 15 minutes, with a relatively clear theme and main line, and a relatively continuous and complete storyline. The subject matter of micro-short dramas is very broad and can meet the needs of different audience groups.

[0056] The traditional short drama production process has the following main shortcomings:

[0057] 1. Long production cycle: From script creation to film release, the entire process takes weeks or even months, and it is impossible to quickly respond to market demand and hot events. 2. High production cost: It involves a lot of human, material and financial investment, including screenwriters, directors, actors, film crews, post-production teams, etc., and the overall cost is high. 3. Limited creativity: Due to factors such as real scenes and actor performances, some creative ideas are difficult to realize and the creative space is limited.

[0058] To solve the above problems, the present invention provides an automated micro-drama generation method based on artificial intelligence generated content (AIGC technology, Artificial Intelligence Generated Content, generative artificial intelligence, using artificial intelligence to generate content, including text, images, videos and audio and other forms), the purpose is to achieve the following effects: 1. Shorten the production cycle: By introducing AIGC technology, the key elements such as storyline, storyboard, key frames, video clips, character dialogues, etc. are automatically generated, greatly shortening the time from creativity to film, so that the micro-drama can quickly respond to market demand and hot events. 2. Reduce production costs: Use artificial intelligence to replace part of the manual work, such as screenwriting, storyboard, animation, dubbing, etc., to reduce the dependence on professionals, thereby significantly reducing the production cost of micro-dramas, making content production more economical and efficient. 3. Expand creative space: With the help of AIGC technology, break the limitations of real scenes and actor performances, generate more diverse and imaginative creative content, and provide unlimited imagination space for micro-drama creation. 4. Empowering ordinary users: Lowering the threshold for short drama creation, allowing ordinary users to easily create short dramas with personality and creativity without professional skills, and achieve self-expression and value realization. 5. Improving production efficiency: By building an integrated and automated short drama generation pipeline, combining various AI agents to form an efficient and smooth production process, the production efficiency and output quality of short dramas can be significantly improved.

[0059] Specifically, the present invention provides an automated short drama production method based on multi-agent AIGC technology, which realizes one-stop creation from story generation to finished product output through a self-developed Pipeline (pipeline, i.e., algorithm process). The system workflow is as follows: 1. The user inputs a creative theme into the system, for example: generating a short drama based on "The Little Rabbit's Forest Adventure" as the material.

[0060] 2. The story generation agent automatically generates a complete storyline based on the creative theme provided by the user. 3. After receiving the storyline, the storyboard generation agent divides it into several scenes and generates a storyboard script for each scene. Each storyboard script contains the commentary, visual description, shot description and atmosphere information of the scene. 4. The commentary information is simultaneously input into the commentary dubbing generation agent and the subtitle generation module. The former is responsible for synthesizing the dubbing audio that matches the commentary content, and the latter is responsible for generating video subtitles synchronized with the commentary. 5. The visual description information is input into the keyframe generation agent, which generates keyframe images that match the description. During the generation process, the image discriminator agent will evaluate the quality of the generated image in real time and decide whether to require the keyframe generation agent to regenerate. 6. The high-quality keyframe images and shot description information are input into the video generation agent together for synthesizing the video clip of the scene. Similarly, the video discriminator agent will evaluate the quality of the generated video and ask the video generation agent to regenerate if necessary. 7. The atmosphere information is input into the background music generation agent, which generates background music that matches the atmosphere of the scene. 8. Finally, align the commentary dubbing, subtitles, video clips and background music of each scene according to the timeline, and splice all the scenes in the order of the storyboard script to output a complete AIGC short play.

[0061] The present invention gives full play to the generation capability and automation advantages of AIGC technology, greatly improves the efficiency and quality of short drama creation, while reducing production costs, and provides an innovative solution for the content creation industry.

[0062] The multi-agent based automatic generation method of short plays described in the preferred embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the multi-agent-based automatic generation method of short plays includes the following steps:

[0063] Step S10, obtaining a user-created theme, and performing storyline generation and storyboard processing according to the user-created theme to obtain a plurality of storyboard scripts.

[0064] The story generation part includes: 1. Determine the basic framework and theme of the story: through user input of creativity or market research, clarify the theme and tone of the story to be generated; 2. Call the story generation agent: Use the Chinese large language model to train the story generation agent, which can generate a complete storyline that matches the theme and tone.

[0065] Specifically, story data and script data are obtained, and the story data and the script data are preprocessed to obtain a training set, wherein the preprocessing includes denoising, format conversion, segmentation and labeling; a preset large language model is determined, and a cross-validation method is used to perform model training and model fine-tuning on the preset large language model according to the training set to obtain the story generation agent.

[0066] Introduction to the algorithm model used by the story generation agent: The Chinese Large Language Model is a large language model (LLM) based on the Transformer neural network architecture. It has mastered complex language patterns and knowledge through training on massive text data. In order to further improve the performance of the Chinese Large Language Model in the field of story generation, the present invention collects a large amount of high-quality story data and script data, and performs targeted Fine-tuning on the Chinese Large Language Model. Through retraining on data in specific fields, the Chinese Large Language Model has learned more professional and detailed story writing skills and knowledge, forming an agent specifically used for story generation (i.e., the story generation agent in the present invention).

[0067] The following is the fine-tuning process of the story generation agent: 1. Data preparation: Collect and organize a large amount of high-quality story data and script data, covering different themes, topics and styles, to ensure the comprehensiveness and diversity of the data; 2. Data cleaning: Clean and preprocess the collected data (preprocessing includes noise removal, format conversion, segmentation and annotation, etc.) to ensure data quality and consistency and prepare for the fine-tuning task; 3. Fine-tuning settings: According to the characteristics of the story generation task, design a suitable fine-tuning scheme (for example, design a suitable generation goal, such as generating characters and plot descriptions, and guiding the model to learn the key points of story writing. For example, carefully design some prompt forms to start the model's story generation), and select the number of fine-tuning layers, learning rate, and Batch. -Size (fine-tuning parameters), number of training rounds and other hyperparameters, as well as determining the training objectives and loss functions; 4. Model fine-tuning: Load the pre-trained Chinese large language model and the prepared story data into the fine-tuning script, start the fine-tuning process, and the model learns the language patterns, writing skills, plot structure and other knowledge in the story data in a self-supervised manner, and fine-tunes the original parameters to adapt to the story generation task; 5. Evaluation and verification: Regularly evaluate the generation effect of the model during the fine-tuning process, use cross-validation and other methods to evaluate the performance of the model in the story generation task, adjust the fine-tuning strategy according to the evaluation results, and optimize the model; 6. Model saving: After the fine-tuning process is completed, save the fine-tuned model parameters for subsequent use and deployment.

[0068] The steps of cross-validation are as follows: 1. Randomly divide the complete data set into k subsets of similar size (usually k=5 or 10); 2. Select k-1 subsets as training sets each time, and the remaining 1 subset as validation set, train the model and evaluate the performance on the validation set; 3. Repeat step 2 k times so that each subset has the opportunity to be used as a validation set; 4. Average the results of the k evaluations as an estimate of the model's performance on the complete data set.

[0069] The steps to adjust the fine-tuning strategy are as follows: 1. Adjust training hyperparameters, such as learning rate, Batch-Size (fine-tuning parameters), number of training rounds, etc., to find the best training configuration; 2. Optimize the loss function design; 3. Expand or optimize the training data to improve data quality; 4. Introduce additional pre-training tasks, such as text classification, sentiment analysis, etc., to allow the model to learn more prior knowledge that is helpful for story writing.

[0070] Acquire a user-created theme, and input the user-created theme into a story generation agent, and perform storyline generation processing on the user-created theme through the story generation agent to obtain a complete storyline; input the complete storyline into a storyboard generation agent, and perform storyboard processing on the complete storyboard through the storyboard generation agent to obtain multiple storyboard scenes and a storyboard script corresponding to each storyboard scene.

[0071] like Figure 3 As shown in the figure, the functions and outputs of the story generation agent are as follows: The story generation agent is the starting point of the AIGC short drama production process. Its main function is to automatically generate a complete and rich storyline based on a simple creative point or idea input by the user. The user only needs to provide a theme, style or key elements of a story, and the story generation agent can automatically build an engaging and complete story outline around this creative point.

[0072] Functions and outputs of the storyboard generation module: The core function of the storyboard generation module is to automatically generate accurate and detailed storyboard descriptions, providing necessary guidance information for the subsequent generation of commentary dubbing, key frame images, video clips, background music and other elements. At the same time, these storyboard scripts use standardized formats and description methods to ensure seamless connection between modules and consistency of information transmission.

[0073] Introduction to the algorithm model used by the storyboard generation module: The storyboard generation module uses large pre-trained language models based on the Transformer architecture, such as GPT-4 or Claude-3. These models have mastered excellent natural language understanding and generation capabilities through pre-training on massive text data. When generating storyboard scripts, the module uses CoT (Chain-of-Thought) Prompt technology to guide the model to make step-by-step reasoning and decisions. By designing reasonable thought chain prompts, the model can generate clear, logical, and detailed storyboard descriptions.

[0074] like Figure 4 As shown in the figure, the functions and outputs of the storyboard generation module: The core function of the storyboard generation module is to automatically generate accurate and detailed storyboard descriptions, providing necessary guidance information for the generation of subsequent commentary dubbing, key frame images, video clips, background music and other elements. At the same time, these storyboard scripts use a standardized format and description method to ensure seamless connection and information transfer between modules.

[0075] Step S20: performing dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles.

[0076] The steps of generating the commentary dubbing include: 1. Receiving the output of the storyboard generation module: extracting the commentary text specifically used for dubbing from the standardized storyboard script as the input of speech synthesis; 2. Text-to-speech: using advanced TTS (Text-to-Speech) technology, the commentary text is automatically converted into a natural, fluent and expressive dubbing voice. The TTS model can generate professional dubbing effects with just the right intonation, pauses and speech speed based on the text content and emotional information. 3. Outputting the dubbing commentary audio: saving the high-quality speech synthesized by TTS in a standard audio file format, such as WAV or MP3.

[0077] Specifically, the commentary is input into a commentary dubbing generation agent and a subtitle generation module respectively; the commentary is subjected to text processing and language feature analysis by the commentary dubbing generation agent to obtain target language features, wherein the text processing includes text normalization, word segmentation and part-of-speech tagging, and the language feature analysis includes grammatical analysis, semantic analysis and prosody prediction; the target language features are input into an acoustic model, acoustic parameters of the target language features are extracted by the acoustic model to obtain target acoustic parameters, and the target acoustic parameters are input into a vocoder to output target dubbing audio.

[0078] Introduction to the algorithm model used by the intelligent agent for generating narration dubbing: Inside the TTS system, the input text will first undergo a series of preprocessing (preprocessing includes text normalization, word segmentation, part-of-speech tagging, etc.) to convert the original text into a standardized text sequence. Then, the system will perform linguistic analysis on these text sequences, such as grammatical analysis, semantic analysis, prosody prediction, etc., to extract various linguistic features related to speech synthesis. Next, the TTS system will input the extracted linguistic features into the acoustic model, which is a machine learning model based on a deep neural network that can learn the complex mapping relationship between text features and speech features. The present invention uses a Seq2Seq model based on the Transformer architecture. The output of the acoustic model is a series of acoustic parameters, such as spectral parameters, fundamental frequency parameters, etc. Finally, the TTS system will input these acoustic parameters into the vocoder to generate a more natural and clear speech output.

[0079] TTS speech synthesis technology makes full use of the powerful capabilities of deep learning. By accurately modeling key elements such as speech rhythm and emotion, it generates highly expressive and persuasive realistic speech. It can not only accurately restore the text content, but also generate emotionally expressive speech output based on the emotional color and semantic information of the text, making the dubbing commentary more vivid and lifelike.

[0080] like Figure 5 As shown in the figure, the functions and outputs of the narration and dubbing generation agent module: The narration and dubbing generation agent is a key module in the AIGC micro-short drama production process. Its main function is to automatically generate high-quality and professional narration and dubbing audio according to the standardized storyboard script output by the storyboard generation module, adding vivid and attractive narration to the video, thereby greatly improving the video's watchability and dissemination effect.

[0081] The subtitle generation module performs subtitle generation processing on the commentary to obtain target video subtitles, wherein the subtitle generation processing includes commentary reception, timeline allocation, subtitle line division, subtitle formatting and subtitle output.

[0082] The implementation steps of the subtitle generation module include: 1. Narration reception: The subtitle generation module first receives the commentary data from the storyboard generation module; 2. Timeline allocation: According to the mainstream commentary speed, the module will allocate a suitable time range for each commentary; 3. Subtitle line division: If a commentary is too long, it may exceed the optimal length of the subtitle to display it at one time. The module will split it into multiple lines of subtitles according to the semantic and grammatical structure; 4. Subtitle formatting: According to the requirements of the target subtitle format, the module will integrate the subtitle text after line division with the timeline information to generate a standardized subtitle file. Taking the SRT format as an example, each subtitle segment needs to contain information such as serial number, start time, end time, subtitle text, etc., and arrange and separate them in a specific format. The module will automatically generate these subtitle attributes and write them into the subtitle file; 5. Subtitle output: The subtitle generation module will output the verified subtitle file for subsequent video synthesis and playback.

[0083] like Figure 6 As shown in the figure, the functions and outputs of the subtitle generation module: The main function of the subtitle generation module is to automatically generate a subtitle file synchronized with the video content based on the commentary data output by the storyboard generation module. The input of this module is the commentary text, and the output is a subtitle file in a standard format, such as SRT. High-quality, synchronized and accurate subtitles can significantly improve the accessibility and user experience of the video. It not only helps the hearing-impaired and non-native audiences better understand the video content, but also provides a more convenient and friendly viewing experience for all audiences. Therefore, the application of the subtitle generation module is an indispensable part of the production of AIGC micro-short dramas. It effectively expands the audience of the video and improves the dissemination effect and influence of the video.

[0084] Step S30: performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip.

[0085] The steps of key frame generation include: 1. Visual information reception: The key frame generation module first receives the visual information description data from the storyboard generation module. These data are provided in a structured form, describing in detail the scene content, character actions, object layout, color style and other visual elements of each shot, providing the necessary conditions and constraints for the generation of key frame images. 2. Consistent image generation: The key frame generation module adopts an advanced consistency diffusion model to generate a series of high-quality, coherent key frame images based on the visual information description. 3. Image quality discrimination: In order to further ensure the visual quality and consistency of the generated key frames, the module introduces an image discriminator based on a multimodal image understanding large model. The discriminator can evaluate the quality of the generated images from multiple dimensions such as content, aesthetics, and semantics, and promptly discover and mark key frames with inconsistent styles, abnormal damage, poor visual effects, etc., and then feed back to the key frame generation agent to regenerate the key frames; 4. Key frame output: The key frame generation module outputs the optimized key frame image sequence.

[0086] Specifically, the visual information is input into a key frame generation agent, and the key frame generation agent performs key frame generation processing on the visual information to obtain a target key frame image; the target key frame image and the lens information are input into a video generation agent, and the video generation agent performs video generation processing based on the target key frame image and the lens information to obtain a target video clip.

[0087] Introduction to the algorithm model used by the keyframe generation agent: The consistent keyframe generation agent uses an advanced story diffusion model (a story visualization tool that converts text, design or photos into visual stories through diffusion model technology), which is an improved variant of the Stable Diffusion model specifically designed to generate coherent image sequences. The story diffusion model gradually generates high-quality, consistent keyframe images from text descriptions through an iterative diffusion process. The model incorporates a consistent attention mechanism that can associate and align the visual features of the current frame with the previous and next frames, ensuring that the generated keyframes remain coherent and consistent in terms of characters, scenes, actions, etc., avoiding problems such as sudden changes in the picture.

[0088] Specifically, given a batch of image features Where B, N and C represent batch-size (fine-tuning parameters), the number of tokens and the number of channels per image, respectively. Define a function Attention(X k ,X q ,X v ) to calculate self-attention. k , X q and X vRepresent the query, key, and value used in the attention calculation. The original self-attention is for each image feature I in I i Independently performed on the image feature I i Projected to Q i ,K i ,V i And send it to the attention function, and get: i =Attention(Q i ,K i ,V i ). In order to establish connections within a batch of images to maintain topic consistency, consistent self-attention samples some tokens S from other image features in the batch. i :

[0089] S i =RandSample(I1,I2,…,I i-1 ,I i+1 ,…,I B-1 ), where RandSample represents the random sampling function. After sampling, the sampled tokenS i With image feature I i Pair to form a new set of tokenP i , then for P i Perform linear projection to generate a new key K Pi Sum value V Pi For consistent self-attention. Here, the original query Q i Remain unchanged. Finally, the self-attention is calculated as follows: i =Attention(Q i ,K Pi ,V Pi ), given a paired token, self-attention is performed on images within a batch to promote interactions between different image features.

[0090] On the basis of generating key frames by the story diffusion model, an image discriminator agent is introduced to further ensure the quality and rationality of the generated images. The discriminator adopts the powerful multimodal image understanding model GPT-4, which has excellent image understanding and analysis capabilities. At the same time, it can also deeply understand the semantics of the image, judge whether the layout and combination of elements in the key frames are reasonable, and whether they conform to common sense logic and text descriptions. GPT-4 can also interpret the inherent meaning of key frames from the perspectives of emotion, aesthetics, culture, etc., and evaluate whether they accurately convey the theme of the story and the artistic style.

[0091] like Figure 7As shown in the figure, the functions and outputs of the key frame generation agent module: by combining the story diffusion model and the GPT-4 discriminator, the consistent key frame generation agent can batch output high-quality, coherent, and semantically rich key frame image sequences, providing a solid visual foundation for the post-production of AIGC micro-short dramas. The application of this technology has greatly improved the picture quality and artistic expression of micro-short drama videos, while also significantly reducing the cost and difficulty of manually selecting key frames, and improving the efficiency and flexibility of content production.

[0092] The steps of video clip generation include: 1. Shot information reception: The video generation agent first receives the shot information description from the storyboard generation module. These data are presented in a structured form, describing the motion trajectory and other elements of the shot in detail, and providing the necessary constraints for the video generation process. 2. Key frame image reception: The video generation agent receives the optimized key frame images from the key frame generation module, which serve as the basic conditions for video generation. 3. Video clip generation: In this step, the agent uses the Video Diffusion technology to generate video clips based on the received key frames and shot information, where the key frame image serves as the starting frame of the video. 4. Video quality judgment: In order to ensure that the generated video clips meet the expected quality, the system introduces a video discriminator based on a large multimodal video understanding model, which can evaluate the overall quality of the video, aesthetic effect, shot accuracy, and compliance with physical rules. The judgment results will be fed back to the video generation agent, and if necessary, the agent will regenerate the video clips based on the feedback to optimize the final output.

[0093] Introduction to the algorithm model used in the video clip generation module: The video clip generation module uses the advanced VideoDiffusion model, which is an extended application of the Diffusion model in the video field. By modeling the time dimension, the Video Diffusion model can generate temporally coherent and spatially consistent dynamic video clips. The model introduces the shot prompt and key frame features extracted by CLIP Encoder in the denoising process, so that the generated video clips can be highly matched with the storyline and visual style. Figure 8 As shown, Figure 8The process of Video Diffusion. The multimodal video understanding model uses the LLaVA-Next-Video (multimodal video large model) model, which is improved based on the LLaVA (multimodal model) architecture. This is a large-scale pre-trained model specifically for video understanding and analysis. LLaVA-Next-Video can understand and judge the visual features of the video from multiple dimensions, accurately identify the key elements in the video, and analyze their attributes, relationships, and interactions. At the same time, the model also has the ability to understand the dynamic information of the video, and can detect and track the timing features such as motion trajectory, action changes, scene transitions, etc. in the video. On this basis, LLaVA-Next-Video can also evaluate the picture quality of the video, detect picture defects, and identify phenomena in the video that violate physical common sense.

[0094] like Fig. 9 As shown in the figure, the functions and outputs of the video generation agent module: The core function of the video generation agent is to automatically create dynamic video clips that match the storyline and have moving visual effects. Through this module, a large number of high-quality, plot-rich video materials can be efficiently generated to provide sufficient dynamic visual content for the final short drama synthesis. In order to further ensure the quality and rationality of the generated video, the video generation agent introduces a video discriminator module, which evaluates the various indicators of the generated video in an automated way, and feeds the evaluation results back to the video generation agent to guide it to continuously adjust and optimize the generation strategy until the generated video reaches the ideal quality level. The combination of the video generation agent and the video discriminator greatly improves the efficiency and quality of AIGC micro-short drama creation and reduces labor and time costs.

[0095] Step S40: performing background music generation processing according to the atmosphere information to obtain target background music.

[0096] The background music generation steps include: 1. Atmosphere information reception: The background music generation agent receives the atmosphere information output by the storyboard generation agent, which is a reference for generating the corresponding atmosphere background music. 2. Background music generation: Call the background music generation agent to generate background music that meets the text description according to the text condition guidance.

[0097] Specifically, the atmosphere information is input into a background music generation agent, and the atmosphere key information in the atmosphere information is extracted through the background music generation agent, wherein the atmosphere key information includes emotional tone, scene characteristics, and character status; the atmosphere key information is subjected to music element conversion processing and music element matching processing to obtain the target background music.

[0098] Introduction to the algorithm model used by the background music generation agent: The present invention uses a generation model called MUSICGEN (Music Generation Model), which is based on the Transformer architecture and can process multiple music representation streams, such as audio tokens. The workflow of the MUSICGEN generation model can be divided into the following steps: 1. The model uses natural language processing (NLP) technology to parse and understand the input text description and extract key information, such as music style, instruments used, emotional tone, etc.; 2. The model converts the music into a series of discrete tokens or notes, making it a form that can be processed by the model. On this basis, MUSICGEN guides the music generation process based on text information to ensure that the generated music matches the text description; 3. The model converts the generated music representation into an actual audio signal to complete the entire text-to-music generation task.

[0099] like Fig.10 As shown in the figure, the functions and outputs of the background music generation agent module are as follows: when generating background music, the agent will comprehensively analyze the key information in the atmosphere description, such as emotional tone, scene characteristics, character status, etc., and convert it into musical elements, such as melody, harmony, rhythm, timbre, etc. By matching and combining these musical elements, the agent can automatically generate background music clips that are highly consistent with the atmosphere description. The generated music echoes the scene atmosphere in terms of emotional expression, style positioning, sound texture, etc., which can effectively set off the emotions of the video, render the story atmosphere, and add artistic appeal to the micro-short drama screen.

[0100] Step S50, the target dubbing audio, the target video subtitles, the target video clips and the target background music are time-aligned and spliced ​​to obtain a target synthesized skit.

[0101] The steps of short play synthesis include: 1. Material integration: In the final stage of AIGC micro short play production, the short play synthesis module will summarize the various materials output by the previous steps, including narration dubbing, subtitle text, video clips and background music. These materials are the basis of short play synthesis; 2. Audio and video fusion: After the material integration is completed, the short play synthesis module begins to perform audio and video fusion processing. Audio and video fusion is performed at the video clip level, that is, each video clip is matched with the corresponding narration dubbing, background music and subtitle text. First, the audio processing technology is used to mix the narration dubbing and background music into the audio track of the video clip, and the audio is edited and optimized according to the volume, sound effects, emotions and other parameters to create a harmonious and unified audio-visual effect. Finally, the short play synthesis module superimposes the corresponding subtitle layer on the video clip according to the subtitle text and timeline to complete the fusion of subtitles and video images. The result of audio and video fusion is a series of video clips with commentary dubbing, background music and subtitles. These clips are the basic units that constitute a complete short drama. 3. Video synthesis: After all video clips have completed audio and video fusion, the short drama synthesis module begins to perform video synthesis tasks. Video synthesis needs to strictly follow the scene sequence, time clues and logical relationships provided by the storyboard script to ensure the coherence and rationality of the short drama narrative. The short drama synthesis module uses video editing technology to connect the video clips end to end and add necessary transition effects at the connection, such as fade in and fade out, screen swiping, erasing, etc., to enhance visual fluency and rhythm.

[0102] Specifically, the target dubbing audio, the target video subtitles, the target video clip and the target background music are input into a skit synthesis module; the target dubbing audio and the target background music are mixed with the target video clip through the skit synthesis module using audio processing technology, and audio editing and audio optimization are performed to obtain a first video clip; the target video subtitles are superimposed on the first video clip according to the timeline to obtain a second video clip; the second video clip is spliced ​​using video editing technology to obtain a target synthesized skit.

[0103] Technical introduction: Audio and video fusion is mainly achieved through audio synthesis and video editing technology. First, the system decodes the audio files of the commentary dubbing and background music and converts them into original audio sampling data. Then, according to the timeline information of the video clip, the audio data is synchronized with the video frame. Next, the system uses an audio mixing algorithm, such as linear mixing, to mix the audio data of the commentary dubbing and background music according to the set volume ratio to form a unified audio track. Finally, the mixed audio track is encoded and synthesized with the video track to generate a new video file in which the audio and video have been precisely synchronized.

[0104] For adding subtitles, text overlay and video encoding technology are used. First, the system will segment the subtitle text according to the timeline to determine the appearance time and duration of each subtitle. Then, the text rendering engine is used to convert the subtitle text into an image and set style attributes such as font and color. Next, the system synchronizes the subtitle image with the video frame according to the timeline of the subtitle to determine the position of the subtitle on each frame. Finally, the video encoder FFmpeg is used to encode the video frame sequence with subtitles into a new video file. During the encoding process, the subtitle image will be superimposed on the video frame and integrated with the video screen.

[0105] like Fig.11 As shown in the figure, the functions and outputs of the short play synthesis module: The short play synthesis module is the end point of the AIGC micro short play production process. Through intelligent video editing and audio processing technologies, such as video splicing, transition effects, audio synthesis, subtitle addition, etc., the video clips, commentary dubbing, background music, subtitles and other materials generated by the previous modules are integrated to automatically generate a complete, smooth, expressive, high-quality micro short play video work, which not only ensures the artistic level of the micro short play, but also greatly improves the production efficiency.

[0106] Beneficial effects of the present invention:

[0107] 1. Significantly shorten the production cycle: Through artificial intelligence, scripts, storyboards, animations, dubbing, etc. are automatically generated to achieve end-to-end fully automated production, shortening the production cycle from several months to a few hours.

[0108] 2. Significantly reduce production costs: It replaces most of the manual links and only requires a small number of technicians to operate, greatly saving labor costs and various expenses.

[0109] 3. Greatly expand the creative space: Using the imagination and generative capabilities of AI, you can create fantasy scenes and visual effects that go beyond the limitations of reality and inspire unlimited creativity.

[0110] 4. Stable output of high-quality content: Through AI models trained on massive amounts of high-quality data, the high level and stability of generated content are guaranteed, reducing dependence on professional and technical personnel.

[0111] 5. Flexible customization of personalized content: Automatically adjust the style and theme of generated content based on user preferences and feedback to provide a personalized skit experience.

[0112] The present invention is an AIGC system that realizes the full-process automated production of micro-short dramas. The problems of long production cycle, high cost, and limited creativity of traditional micro-short dramas have been perfectly solved in the present invention. Through the collaborative work of multiple professional AI agents, the present invention can automatically complete the entire process from storyline generation, storyboard design, audio-visual material synthesis to the final short drama generation according to the creative theme input by the user, greatly improving the production efficiency and creative freedom of micro-short dramas. The present invention includes functional modules such as story generation, storyboard generation, commentary dubbing generation, subtitle generation, key frame generation, image discrimination, video generation, video discrimination, and background music generation, realizing the full-process automation of micro-short drama production.

[0113] The present invention innovatively introduces image discriminator agents and video discriminator agents to form a "generation-discrimination" recursive optimization mechanism. In the key frame generation and video clip synthesis stages, the discriminator agent evaluates the quality of the generated content and decides whether to require the corresponding generation agent to regenerate until the quality meets the requirements. This mechanism efficiently replaces the content screening stage that requires manual participation in traditional solutions, which not only improves the generation efficiency, but also ensures the high quality and consistency of the generated content.

[0114] The present invention adopts a modular architecture design, and each agent interacts with each other through a standardized interface. This decoupled design allows each agent to independently upgrade algorithms and optimize performance, and the entire system has the ability to continuously evolve and self-improve. At the same time, this architecture also facilitates the introduction of more types of content generation agents in the future, expanding the system's creative capabilities.

[0115] The focus of this invention is on the innovation of the micro-short drama production process and paradigm. By deeply integrating artificial intelligence technology with micro-short drama creation, it has created a new micro-short drama generation paradigm, which is expected to lead the development direction of future micro-short drama content production and provide more possibilities for ordinary users to participate in cultural and creative production.

[0116] Furthermore, if Fig.12 As shown, based on the above-mentioned multi-agent-based automatic generation method of micro-short plays, the present invention also provides a multi-agent-based automatic generation system of micro-short plays, wherein the multi-agent-based automatic generation system of micro-short plays includes:

[0117] A storyboard script generation module 51 is used to obtain a user-created theme, and perform storyline generation and storyboard processing according to the user-created theme to obtain multiple storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information, and atmosphere information;

[0118] An audio subtitle generation module 52 is used to perform dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles;

[0119] A video segment generation module 53 is used to perform key frame generation processing according to the visual information to obtain a target key frame image, and perform video generation processing according to the target key frame image and the lens information to obtain a target video segment;

[0120] A background music generation module 54, used to generate background music according to the atmosphere information to obtain target background music;

[0121] The synthetic skit generation module 55 is used to perform timeline alignment and splicing processing on the target dubbing audio, the target video subtitles, the target video clips and the target background music to obtain a target synthetic skit.

[0122] Furthermore, if Fig.13 As shown, based on the above-mentioned multi-agent-based micro-drama automatic generation method and system, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Fig.13 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0123] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as a hard disk or memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a multi-agent-based micro-drama automatic generation program 40 is stored on the memory 20, and the multi-agent-based micro-drama automatic generation program 40 can be executed by the processor 10, thereby realizing the multi-agent-based micro-drama automatic generation method in the present application.

[0124] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the multi-agent-based automatic generation method of micro-short plays.

[0125] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0126] In one embodiment, when the processor 10 executes the multi-agent based mini-drama automatic generation program 40 in the memory 20, the steps of the multi-agent based mini-drama automatic generation method as described above are implemented.

[0127] In summary, the present invention provides a method, system and terminal for automatically generating micro-short plays based on multi-agents, the method comprising: obtaining a user-created theme, and performing plot generation processing and storyboard processing according to the user-created theme to obtain multiple storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information and atmosphere information; performing dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles; performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip; performing background music generation processing according to the atmosphere information to obtain target background music; performing timeline alignment processing and splicing processing on the target dubbing audio, the target video subtitles, the target video clip and the target background music to obtain a target synthetic short play. Through the collaborative work of multiple professional AI agents, the present invention can automatically complete the entire process from storyline generation, storyboard design, audio-visual material synthesis to the final short play generation according to the creative theme input by the user, greatly improving the production efficiency and creative freedom of micro-short plays, and realizing the full process automation of micro-short play production.

[0128] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0129] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0130] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-agent based automatic generation method for short plays, characterized in that: The multi-agent-based automatic generation method for short plays includes: Acquire a user-created theme, and perform storyline generation and storyboard processing according to the user-created theme to obtain a plurality of storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information, and atmosphere information; Perform dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles; Performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip; Perform background music generation processing according to the atmosphere information to obtain target background music; The target dubbing audio, the target video subtitles, the target video clips and the target background music are subjected to time axis alignment and splicing processing to obtain a target synthetic skit.

2. The multi-agent based automatic generation method of short plays according to claim 1 is characterized in that: The obtaining of the user creation theme, and performing storyline generation and storyboard processing according to the user creation theme to obtain multiple storyboard scripts specifically includes: Acquire a user-created theme, and input the user-created theme into a story generation agent, and perform storyline generation processing on the user-created theme by the story generation agent to obtain a complete storyline; The complete storyline is input into a storyboard generation agent, and the storyboard generation agent is used to storyboard the complete storyline to obtain a plurality of storyboard scenes and a storyboard script corresponding to each storyboard scene.

3. The method for automatically generating short plays based on multi-agents according to claim 2 is characterized in that: The method of obtaining a user-created theme, and performing storyline generation and storyboard processing according to the user-created theme to obtain a plurality of storyboard scripts, further includes: Acquire story data and script data, and pre-process the story data and the script data to obtain a training set, wherein the pre-processing includes denoising, format conversion, segmentation, and annotation; A preset large language model is determined, and a cross-validation method is used to perform model training and model fine-tuning on the preset large language model according to the training set to obtain the story generation agent.

4. The method for automatically generating short plays based on multi-agents according to claim 1 is characterized in that: The dubbing conversion process and subtitle generation process are performed according to the commentary to obtain target dubbing audio and target video subtitles, specifically including: Input the commentary into the commentary dubbing generation agent and the subtitle generation module respectively; The commentary dubbing generating agent performs text processing and language feature analysis on the commentary to obtain target language features, wherein the text processing includes text normalization, word segmentation and part-of-speech tagging, and the language feature analysis includes grammatical analysis, semantic analysis and prosody prediction; Inputting the target language features into an acoustic model, extracting acoustic parameters of the target language features through the acoustic model to obtain target acoustic parameters, and inputting the target acoustic parameters into a vocoder to output target dubbing audio; The subtitle generation module performs subtitle generation processing on the commentary to obtain target video subtitles, wherein the subtitle generation processing includes commentary reception, timeline allocation, subtitle line division, subtitle formatting and subtitle output.

5. The method for automatically generating short plays based on multi-agents according to claim 1 is characterized in that: The step of performing key frame generation processing according to the visual information to obtain a target key frame image, and performing video generation processing according to the target key frame image and the lens information to obtain a target video clip, specifically includes: Inputting the visual information into a key frame generation agent, and performing key frame generation processing on the visual information by the key frame generation agent to obtain a target key frame image; The target key frame image and the lens information are input into a video generation agent, and the video generation agent performs video generation processing according to the target key frame image and the lens information to obtain a target video clip.

6. The multi-agent based automatic generation method of short plays according to claim 1 is characterized in that: The generating process of background music according to the atmosphere information to obtain target background music specifically includes: Inputting the atmosphere information into a background music generation agent, and extracting key atmosphere information from the atmosphere information through the background music generation agent, wherein the key atmosphere information includes emotional tone, scene characteristics, and character status; The atmosphere key information is subjected to music element conversion processing and music element matching processing to obtain target background music.

7. The multi-agent based automatic generation method of short plays according to claim 1 is characterized in that: The target dubbing audio, the target video subtitles, the target video clips and the target background music are subjected to time axis alignment and splicing processing to obtain a target synthesized skit, specifically comprising: Inputting the target dubbing audio, the target video subtitles, the target video clips and the target background music into a skit synthesis module; The target dubbing audio and the target background music are mixed with the target video clip by the skit synthesis module using audio processing technology, and audio editing and audio optimization are performed to obtain a first video clip; Superimposing the target video subtitles onto the first video segment according to the timeline to obtain a second video segment; The second video clips are spliced ​​using video editing technology to obtain a target synthetic short drama.

8. A multi-agent based automatic generation system for short plays, characterized in that: The multi-agent-based short drama automatic generation system includes: A storyboard script generation module is used to obtain a user-created theme, and perform storyline generation and storyboard processing according to the user-created theme to obtain multiple storyboard scripts, wherein the storyboard scripts include commentary, visual information, lens information, and atmosphere information; An audio subtitle generation module is used to perform dubbing conversion processing and subtitle generation processing according to the commentary to obtain target dubbing audio and target video subtitles; A video clip generation module, configured to perform key frame generation processing according to the visual information to obtain a target key frame image, and perform video generation processing according to the target key frame image and the lens information to obtain a target video clip; A background music generation module, used to generate background music according to the atmosphere information to obtain target background music; The synthetic skit generation module is used to perform timeline alignment and splicing processing on the target dubbing audio, the target video subtitles, the target video clips and the target background music to obtain a target synthetic skit.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a multi-agent based micro-short play automatic generation program stored in the memory and executable on the processor. When the multi-agent based micro-short play automatic generation program is executed by the processor, the steps of the multi-agent based micro-short play automatic generation method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multi-agent-based automatic generation program for micro-short plays, and when the multi-agent-based automatic generation program for micro-short plays is executed by a processor, the steps of the multi-agent-based automatic generation method for micro-short plays as described in any one of claims 1 to 7 are implemented.

Images